J 


® 


Europaisches Patentamt 
European Patent Office 
Office europton das brevets 



(fi) Publication number: 0 644 476 A2 


12) 


EUROPEAN PATENT APPLICATION 


@ Application number : 943059444 
@ Date of filing: 11.08,94 


si) int. ci. 6 : G06F 3/033 


® Priority: 11.08.93 JP 198707/93 

© Date of publication of application : 
22.03.95 Bulletin 95/12 

(g) Designated Contracting States : 
DE FR GB 

© Applicant: SONY CORPORATION 
7-35, Kltashlnagawa 6-chome 
Shlnagawa-ku 
Tokyo (JP) 


@ Inventor: Shlnozuka, Hldeakl, c/o Sony 
Corporation 

7-35 Kitashinagawa 6-chome, 

Shinagawa-ku 

Tokyo (JP) 

Inventor: Chatanl, Masayukl, do Sony 
Coiporation 

7-35 Kltashlnagawa 6-chome, 

Shinagawa-ku 

Tokyo (JP) 

(?4) Representative : Nicholla, Michael John 
JA KEMP & CO. 
14, South Square 
Gray's Inn 

London WC1R 5LX (GB) 


(g) Method of changing display condition of handwritten Input 

(5?) A predetermined relationship is established 
with respect to an arbitrary shape of a pattern, 
or a character, which is inputted by employing a 
pen on an input table, and displayed on a 
display unit in a handwriting input display ap- 
paratus. Further, a gesture is handwritten on 
the input tablet with the pen, so that a preselec- 
ted process operation is canned out for the 
shape previously represented on the display 
unit, and therefore the display condition thereof 
may be changed. The line segment under repre- 
sentation is quickly and readily changed into an 
arbitrary condition. When a gesture is entered 
with respect to a plurality of line segments, 
these line segments are rearranged in response 
to a command corresponding to this gesture. 
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The present invention relates to a handwriting in- 
put display method suitable for, e.g., an electronic 
notebook apparatus and a portable computer. 

Very recently, a CAD system has grown, and 
then an arbitrary pattern, or figure may be relatively 5 
simply input In this CAD system, when a pattern hav- 
ing an arbitrary shape is drawn by using, for example, 
a mouse or a pen, a pattern corresponding to this in- 
put pattern is displayed on a monitor. Then, for exam- 
ple, after a plurality of line segments have been en- 10 
tered, the preselected parameters are designated in 
order to change the display conditions of these line 
segments. 

For instance, when a plurality of line segments 
elongated along the vertical direction are displayed in 15 
such a manner that the upper ends thereof are 
aligned along the horizontal direction, the below-men- 
tioned parameters are entered: 

(1) A designation is made of a line segment of in- 
terest 

(2) A designation is made of reference points for 
the respective line segments of interest. 

(3) A straight line such that the reference points 
for the respective line segments of interest 
should be aligned is entered. 

(4) A designation is made as to whether or not an 
interval between the reference points is adjusted. 

(5) A designation is made of the positions of the 
reference points at both ends after alignment 

(6) A designation is made of a method for trans- 
porting a line segment of interest (for example, a 
designation is made as to whether the line seg- 
ment is transported along the y-axis direction, or 
along the direction of the line segment). 
Otherwise, a predetermined mode is selected by 

way of the menu operation, and the process opera- 
tions for the respective line segments are wholly des- 
ignated. 

However, the above-described parameter desig- 
nating method owns such problems that the lengthy 
operating procedure is required and lengthy process 
time is needed. There is another problem that since 
the parameter designation is carried out by entering 
the numeral values, this parameter designation can- 
not be intuitively performed, resulting in poor oper- 
ability. 

Furthermore, the menu operation method owns 
the following drawbacks: 

(1) It is difficult to designate an arbitrary point 
and an arbitrary direction at high speed. 

(2) It is difficult to simply utilize the featured 
amounts of the established line segments (for ex- 
ample, an arbitrary point on a line segment of in- 
terest, a direction, or a length of a line segment 
of interest). 

(3) When the numeral value is entered to desig- 
nate the featured amount, it is difficult to intuitive- 
ly input the featured amount 


The present invention has been made to solve 
these problems, and therefore has an object to pro- 
vide such a method for performing an input process 
of a shape more quickly and more simply. 

A handwriting input display method as described 
in claim 1 is characterized in that a plurality of line 
segments are displayed, a gesture is input so as to 
establish a predetermined relationship with said line 
segments; and display conditions of said line seg- 
ments are changed in response to said gesture. 

A handwriting input display method as described 
in claim 2 is characterized in that a plurality of shapes 
are displayed; a gesture is input so as to establish a 
predetermined relationship with said stapes; a fea- 
tured amount of said gesture is detected; and display 
conditions of said shapes are changed in accordance 
with the featured amount of said gesture. 

This gesture may contain a handwritten charac- 
ter. Also, characters may be aligned by this entered 
gesture. 

In accordance with the handwriting input display 
method as recited in claim 1 , when the gesture is in- 
put to establish a predetermined relationship with the 
line segment the display conditions of the line seg- 
ment are changed in such a manner that the interval 
between the line segments is aligned in response to 
this input gesture. As a result, the display condition of 
the line segment can be simply and quickly changed. 

In accordance with the handwriting input display 
method as recited in claim 2, when the gesture is in- 
put to establish a predetermined relationship with the 
shape such as the character and the pattern, for in- 
stance, the shapes are aligned with an equidistant in- 
terval at the input position of this gesture in response 
to the featured amount of this gesture. As a result, it 
is possible to easily and quickly change the display 
condition of the shape. 

The present invention will be further described 
hereinafter, with reference to the following description 
of exemplary embodiments and the accompanying 
drawings, in which:- 

Fig. 1 is a schematic block diagram for showing 
an arrangement of an apparatus according to an 
embodiment of the present invention, to which 
the handwriting input display method of the pres- 
ent invention has been applied; 
Fig. 2 is a functional block diagram for explaining 
operations of the embodiment of Fig. 1; 
Fig. 3 is another functional block diagram for ex- 
plaining the operation of the embodiment shown 
in Fig. 1; 

Fig. 4 is a flow chart for explaining the operation 
of the embodiment of Fig. 1 ; 
Fig. 5 is a flow chart subsequent to Fig. 4; 
Fig. 6 is a flow chart subsequent to Fig. 5; 
Fig. 7 is a flow chart subsequent to Fig. 6; 
Fig. 8 is a flow chart subsequent to a flow chart 
subsequent to Fig. 7; 
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Fig. 9 is a first diagram for explaining a gesture 
operation to align line segments at an equidistant 
interval; 

Fig. 10 is a second diagram for explaining a ges- 
ture operation to align line segments at an equi- 5 
distant interval; 

Fig. 11 is a third diagram for explaining a gesture 
operation to align line segments at an equidistant 
interval; 

Fig. 12 is a fourth diagram for explaining a ges- 10 
ture operation to align line segments at an equi- 
distance interval; 

Fig. 13 is a fifth diagram for explaining a gesture 
operation to align line segments at an equidistant 
interval; is 
Fig. 14 is a flow chart for explaining in detail the 
operation from the step S66 of Fig. 7 in case 
when the process operation shown in Fig. 9 to 
Fig. 13 is performed; 

Fig. 1 5 is a flow chart for explaining in detail the 20 
operation from the step S68 of Fig. 7 in case 
when the process operation shown in Fig. 9 to 
Fig. 13 is performed; 

Fig. 16 is a flow chart for explaining in detail the 
operation from the step S81 of Fig. 8 in case 25 
when the process operation shown in Rg. 9 to 
Fig. 13 is performed; 

Fig. 17 is a flow chartfor explaining in detailed the 
operation from the step S82 of Fig. 8 in case 
when the process operation shown in Fig. 9 to so 
Fig. 13 is performed; 

Fig. 18 is a first diagram for describing a gesture 
to vertically set two straight lines; 
Fig. 19 is a second diagram for describing a ges- 
ture to vertically set two straight lines; ss 
Fig. 20 is a third diagram for describing a gesture 
to vertically set two straight lines; 
Fig. 21 is a fourth diagram for describing a ges- 
ture to vertically set two straight lines; 
Fig. 22 is a fifth diagram for describing a gesture 40 
to vertically set two straight lines; 
Fig. 23 is a flow chart for explaining in detail the 
process operation from the step S66 of Fig. 7 in 
case when the embodiment shown in Fig. 18 to 
Fig. 22 is performed; 45 
Fig. 24 is a flow chart for explaining in detail the 
process operation from the step S68 of Fig. 7 in 
case when the embodiment shown in Fig. 18 to 
Fig. 22 is performed; 

Fig. 25 is a flow chart for explaining in detail the so 
process operation from the step S8 of Fig. 8 in 
case when the embodiment shown in Fig. 18 to 
Fig. 22 is performed; 

Fig. 26 is a flow chart for explaining in detail the 
process operation from the step S82 of Fig. 8 in 55 
case when the embodiment shown in Fig. 18 to 
Fig. 22 is performed. 

Fig. 27 is a first diagram for describing a gesture 


which aligns one edge points of straight lines on 
a straight line; 

Fig. 28 is a second diagram for describing a ges- 
ture which aligns one edge points of straight lines 
on a straight line; 

Fig. 29 is a third diagram for describing a gesture 
which aligns one edge points of straight lines on 
a straight line; 

Fig. 30 is a fourth diagram for describing a ges- 
ture which al igns one edge points of straight I ines 
on a straight line; 

Fig. 31 is a fifth diagram for describing a gesture 
which aligns one edge points of straight lines on 
a straight line; 

Fig. 32 is a flow chart for explaining in detail the 
process operation from the step S66 of Fig. 7 in 
case that the embodiment shown in Fig . 27 to Fig. 
31 is performed; 

Fig. 33 is a flow chart for explaining in detail the 
process operation from the step S68 of Fig. 7 in 
case that the embodiment shown in Fig. 27 to Fig. 
31 is performed; 

Fig. 34 is a flow chart for explaining in detail the 
process operation from the step S81 of Fig. 8 in 
case that the embodi ment shown in Fig. 27 to fig. 
31 is performed; 

Fig. 35 is a flow chart for explaining in detail the 
process operation from the step S82 of Fig. 8 in 
case that the embodi ment shown in Fig. 27 to fig. 
31 is performed; 

Fig. 36 is a first diagram for explaining a gesture 
to align a plurality of line segments at a predeter- 
mined angle; 

Fig . 37 is a second diagram for explaining a ges- 
ture to align a plurality of line segments at a pre- 
determined angle; 

Fig. 38 is a third diagram for explaining a gesture 
to al ig n a plurality of line segments at a predeter- 
mined angle; 

Fig. 39 is a fourth diagram for explaining a ges- 
ture to align a plurality of line segments at a pre- 
determined angle; 

Fig. 40 is a fifth diagram for explaining a gesture 
to align a plurality of line segments at a predeter- 
mined angle; 

Fig. 41 is a flow chart for describing in detail the 
process operation from the step S66 of Fig. 7 in 
case when the embodiment shown in Fig. 36 to 
Fig. 40 is performed; 

Fig. 42 is a flow chart for describing in detail the 
process operation from the step S68 of Fig. 7 in 
case when the embodiment shown in Fig. 38 to 
Fig. 40 is executed; 

Fig. 43 is a flow chart for describing in detail the 
process operation from the step S81 of Fig. 6 in 
case when the embodiment shown in Fig. 36 to 
Fig. 40 is performed; 

Fig. 44 is a flow chart for describing in detail the 
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process operation from the step S82 of Fig. 8 in 
case when the embodiment shown in Fig. 38 to 
Fig. 40 is executed; 

Fig. 45 is a first diagram for describing a gesture 
to align both of edge points of plural straight lines 5 
on a preselected straight line; 
Fig. 46 is a second diagram for describing a ges- 
ture to align both of edge points of plural straight 
lines on a preselected straight line; 
Fig. 47 is a third diagram for describing a gesture 10 
to align both of edge points of plural straight lines 
on a preselected straight line; 
Fig. 48 is a fourth diagram for describing a ges- 
ture to align both of edge points of plural straight 
lines on a preselected straight line; 15 
Fig. 49 is a flow chart for explaining in detail the 
process operation from the step S66 of Fig. 7 in 
case when the embodiment shown in Fig. 45 to 
Fig. 48 is performed; 

Fig. 50 is a flow chart for explaining in detail the 20 
process operation from the step S68 of Fig. 7 in 
case when the embodiment shown in Fig. 45 to 
Fig. 48 is performed; 

Fig. 51 is a flow chart for explaining in detail the 
process operation from the step S81 of Fig. 8 in 25 
case when the embodiment shown in Fig. 45 to 
Fig. 48 is performed; 

Fig. 52 is a flow chart for explaining in detail the 
process operation from the step 682 of Fig. 8 in 
case when the embodiment shown in Fig. 45 to 30 
fig. 48 is performed; 

Fig. 53 is a first diagram for describing a gesture 
to arrange two straight lines at a predetermined 
angle; 

Fig. 54 is a second diagram for describing a ges- 35 
ture to arrange two straight lines at a predeter- 
mined angle; 

Fig. 55 is a third diagram for describing a gesture 
to arrange two straight lines at a predetermined 
angle; 40 
Fig. 56 is a fourth diagram for describing a ges- 
ture to arrange two straight lines at a predeter- 
mined angle; 

Fig. 57 is a flow chart for describing in detail the 
process operation from the step S66 of Fig. 7 in 45 
case when the embodiment shown in Fig. 53 to 
Fig. 58 is performed; 

Fig. 58 is a flow chart for describing in detail the 
process operation from the step S68 of Fig. 7 in 
case when the embodiment shown in Fig. 53 to 50 
Fig. 56 is executed. 

Fig. 59 is a flow chart for describing in detail the 
process operation from the step S64 of Fig. 8 in 
case when the embodiment shown in Fig. 53 to 
Fig. 56 is performed; 55 
Fig. 60 is a flow chart for describing in detail the 
process operation from the step S82 of Fig. 8 in 
case when the embodiment shown in Fig. 53 to 


Fig. 56 is executed; 

Fig. 61 is a first embodiment for explaining a ges- 
ture to wholly process a plurality of gestures; 
Fig. 62 is a second embodiment for explaining a 
gesture to wholly process a plurality of gestures; 
Fig. 63 is a third embodiment for explaining a ges- 
ture to wholly process a plurality of gestures; 
Fig. 64 is a fourth embodiment for explaining a 
gesture to wholly process a plurality of gestures; 
Fig. 65 is a fifth embodiment for explaining a ges- 
ture to wholly process a plurality of gestures; 
Fig . 66 is a flow chart of a process operation exe- 
cuted subsequent to the flow chart shown in Fig. 
4 to Fig. 6 in case that when the embodiment in- 
dicated in Fig. 61 to Fig. 64 is performed; 
Fig. 67 is a flow chart subsequent to Fig. 66; 
Fig. 68 is a first diagram for explaining a gesture 
of a handwritten character 
Fig. 69 is a second diagram for explaining a ges- 
ture of a handwritten character, 
Fig. 70 is a third diagram for explaining a gesture 
of a handwritten character 
Fig. 71 is a fourth diagram for explaining a ges- 
ture of a handwritten character; 
Fig. 72 is a fifth diagram for explaining a gesture 
of a handwritten character 
Fig. 73 is a flow chart for describing in detail the 
process operation from the step S66 of Fig. 7 in 
case when the embodiment shown in Fig. 68 to 
Fig. 72 is performed; 

Fig. 74 is a flow chart for describing in detail the 
process operation from the step S82 of Fig. 8 in 
case when the embodiment shown in Fig. 68 to 
Fig. 72 is executed; 

Fig. 75 is a further functional block diagram for 
explaining the operation of the embodiment 
shown in Fig. 1; 

Fig. 76 is a first diagram for describing a process 
operation to align characters; 
Fig. 77 is a second diagram for describing a proc- 
ess operation to align characters; 
Fig. 78 is a third diagram for describing a process 
operation to align characters; 
Fig. 79 is a fourth diagram for describing a proc- 
ess operation to align characters; 
Fig. 80 is a fifth diagram for describing a process 
operation to align characters; 
Fig. 81 is a flow chart for describing in detail the 
process operation from the step S66 to Fig. 7 in 
case when the embodiment shown in Fig. 76 to 
Fig. 80 is performed; 

Fig. 82 is a flow chart for describing in detail the 
process operation from the step S68 of Fig. 7 in 
case when the embodiment shown in Fig. 76 to 
Fig. 80 is executed; 

Fig. 83 is a flow chart for describing in detail the 
process operation from the step S81 of Fig. 8 in 
case when the embodiment shown in Fig. 76 to 
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Fig. 80 is performed; and 
Fig. 84 is a flow chart for describing in detail the 
process operation from the step S82 of Fig. 8 in 
case when the embodiment shown in Fig. 76 to 
Fig. 80 is executed. 5 
Fig. 1 is a schematic block diagram for represent- 
ing an arrangement of an apparatus to which a hand- 
writing input display method according to an embodi- 
ment of the present invention has been applied. A 
CPU 1 controls the respective circuit units in accor- 10 
dance with a program previously stored in a ROM 2 
to execute a character input/display operation. In a 
RAM 3, predetermined data and the like are stored, 
if required. To a backup RAM 4, predetermined power 
is continuously supplied from a battery 5, and irrfor- 15 
mation desired to be stored is stored even when the 
power supply of the apparatus is turned OFF. A timer 
6 is so designed to be continuously operable by the 
battery 5 on the basis of a crystal oscillator 7. 

A display controller 11 expands data entered 
therein via a bus line 8 to a V-RAM 12, and causes the 
expanded image to be displayed on a display unit 1 3. 
An input tablet 15 outputs the entered coordinate pos- 
ition data to a tablet interface 14 by operating a pen 
16. The tablet interface 14 outputs this data via the 
bus line 8 to either the CPU 1 , or the display controller 
11. 

A microphone 1 8 outputs a collected speed signal 
to a solid-state recording process circuit 17 so as to 
be recorded thereon. The speed signal outputted 
from the solid-state recording process circuit 17 is 
suppl ied via an adder 20 to a speaker 21 , thereby pro- 
ducing speech. A dialler 19 is under control by the 
CPU 1 to output a predetermined dialling signal via the 
adder 20 to the speaker 21 . 

A plurality of key switch groups 22 are operated 
when the operation of the apparatus is switched, and 
signals responding to the key operations are supplied 
from a key interface 23 via the bus line 8 to the CPU 
1 . A power supply circuit 25 may be turned ON to sup- 
ply the power to the respective circuit units, and such 
various operations by the solid-state recording proc- 
ess circuit 17 as recording, reproducing, stop, and vol- 
ume controls may be performed by operating a pre- 
determined key switch group among the plural key 
switch groups 24. 

Furthermore, an IC card (not shown) may be in- 
serted into a socket 26. An IC card interface 72 trans- 
fers and receives data with this IC card. 

To a pin connector 29, for instance, a modem may 
be connected, and a signal is transferred/received via 
the modem and an extension interface 28. 

It should be noted that the input tablet 15 is con- 
structed of a transparent material and formed on a 
display unit 1 3. Asa result, a representation of the dis- 
play unit 1 3 may be observed through the in put tablet 
15. 

Before describing operations of the embodiment 


shown in Fig. 1, process operations executed by the 
CPU 1 are classified into functional blocks which will 
be explained with reference to Fig. 2 and Fig. 3. This 
CPU 1 executes process operations indicated in flow 
charts of Fig. 4 to Fig. 8. The functional block of Fig. 
2 represents such a functional block related to the 
process operations forjudging the gestures indicated 
in the flow charts of Fig. 4 through Fig. 6. The func- 
tional block of Fig. 3 indicates a functional block for 
executing the process operations corresponding to 
the gestures shown in Fig. 7 and Fig. 8. 

In this embodiment, a display condition of a line 
segment is changed by inputting a gesture in such a 
manner that a predetermined relationship is estab- 
lished with respect to the displayed line segment The 
respective storage units and data base are formed in 
the RAM 3, or the ROM 2 of Fig. 1 , and other portions 
are formed inside the CPU 1 in these functional 
blocks. 

When a gesture having a preselected shape is in- 
putted on an input tablet 15 by operating the pen 16, 
coordinate data thereof which is detected by the input 
tablet 1 5 is entered via the tablet interface 14 into the 
CPU 1 . The CPU 1 supplies this data to an input hand- 
writing memory unit 51 and stores it into this memory 
u nit 51 . When the entered gesture is arranged by not 
a single image, by a plurality of images, the data 
about the images until all of the images for constitut- 
ing tii is gesture are entered are stored into a plural im- 
age handwriting memory 52. 

• A featured handwriting amount memory unit 53 
extracts a featured amount of handwriting stored in 
the input handwriting memory unit 51. This featured 
amount corresponds to, for instance, a direction and 
a length of a straight line portion of a line segment, a 
radius and a centre angle of an arc, a quantity of a 
bending angle (vertex), and the number of loops. This 
featured amount is the same as that of the gesture 
stored in the single image gesture data base 57 and 
the plural image gesture data base 58. 

The featured amount extracted by the handwrit- 
ing featured amount extracting unit 53 is supplied to 
the input featured handwriting amount memory unit 
54 so as to be stored therein. The featured amounts 
existing until all of the images are inputted in case 
that the gestures of the plural images are entered, are 
supplied to a plural image featured handwriting 
amount memory unit 55 so as to be stored therein. 

A handwriting comparing unit 56 compares the 
featured amount of the gesture stored in either the in- 
put featured handwriting amount memory unit 54 or 
the plural image featured handwriting amount mem- 
ory unit 55 with the featured amount of the single im- 
age gesture stored in the single image gesture data 
base 57. In this single image gesture data base 57, a 
combination among a featured amount and an iden- 
tification number of a gesture command constructed 
by a single image (single image gesture), a retrieving 


25 


30 


35 


40 


45 


50 


9 


EP 0 644 476 A2 


10 


method for a line segment to be processed, and a 
process method to be executed is employed as one 
data base element 

The handwriting comparing unit 58 compares the 
featured amount stored in the input featured hand- 5 
writing amount memory unit 54 with the featured 
amount stored in the single image gesture data base 
57 to obtain a resemblance degree of the inputted 
handwriting. A subject changing unit 61 compares the 
resemblance degree of the inputted handwriting sup- 10 
plied from the handwriting comparing unit 56 with a 
preset reference value, and when this resemblance 
degree is greater than the reference value, the input- 
ted handwriting is recog nized as the gesture and then 
both the gesture number thereof and this resem- 15 
blanoe degree are supplied to an established gesture 
subject list memory unit 62 so as to be stored therein. 

On the other hand, in the plural image gesture 
data base 58, a combination among an identification 
number of a plural image gesture, the number of in> 20 
ages for constituting a gesture, a featured amount of 
each image, a featured amount between the images 
(e.g., positional relationship), a retrieving method of 
a line segment under process, and a process method 
to be executed is employed as a data base element 25 
A subject image selecting unit 59 selects predeter- 
mined plural image gestures stored in this plural im- 
age gesture data base 58, and also selects one of 
these plural image gestures for comparison, which 
will then be supplied to the handwriting comparing 30 
unit 56. The handwriting comparing unit 56 compares 
this comparison subject with the one image featured 
amount of the inputted handwriting supplied from the 
input featured handwriting amount memory unit 54, 
thereby obtain the resemblance degree between 35 
them. The subject changing unit 61 recognizes such 
a resemblance degree as a point of a gesture when 
the compared resemblance degree is greater than the 
preset reference value, and supplies this resem- 
blance degree to a plural image gesture subject list 40 
memory unit 60. Then, the supplied resemblance de- 
gree is stored together with the gesture-number 
thereof are stored in this memory unit 60. 

When there are other images than the image 
stored in the plural image gesture subject list memory 45 
unit 60, which constitute this gesture, a subject image 
selecting unit 59 reads out these images from the plu- 
ral image gesture data base 56, and supplies them to 
the handwriting comparing unit 58 as the comparison 
subject. so 

Subsequently, the handwriting comparing unit 56 
compares the thus inputted comparison subject with 
the featured amount of the inputted handwriting sup- 
plied from the input featured handwriting amount 
memory unit 54 to obtain a resemblance degree 55 
therebetween. When the newly inputted resemblance 
degree is greater than a preset reference value, the 
subject changing unit 61 averages this resemblance 


degree and the resemblance degree (point of ges- 
ture) of the previously inputted handwriting, and then 
uses the averaged degree as a new point of the ges- 
ture to update the subject 

As described above when the plural image ges- 
ture having the point higher than the reference value 
is detected in the subject changing unit 61, this ges- 
ture is supplied tot he established gesture subject list 
memory unit 62 and stored therein. 

It should be noted that although the comparison 
is carried out every time the inputted handwriting for 
the single image is entered in the handwriting unit 56, 
when the handwriting is inputted for the plural image 
gesture, a plurality of handwriting may be combined 
for comparison purposes. Alternatively, a plurality of 
images may be sequentially selected for a single im- 
age inputted handwriting, and the image having the 
highest resemblance degree may be selected. 

On the other hand, when the gesture is stored in 
the established gesture subject list memory unit 62 in 
such a manner as described above, a process oper- 
ation corresponding to this gesture is carried out in 
the functional block indicated in Fig. 3. In this func- 
tional block, an established gesture subject list mem- 
ory unit 73 corresponds to the established gesture 
subject list memory unit 62 shown in Fig. 2. Similarly, 
a gesture data base 72 corresponds to the single im- 
age gesture data base 57 and the plural image ges- 
ture data base 56 shown in Fig. 2, and also a featured 
handwriting amount memory unit 74 corresponds to 
the input featured handwriting amount memory unit 
54 and the plural image featured handwriting amount 
memory unit 55 indicated in Fig. 2. 

A process determining u nit 75 reads such data as 
a method for retrieving a line segment interest, a 
method for ext racting a featured amount of a I ine seg- 
ment of interest, and a method for processing a line 
segment of interest corresponding to the gesture 
stored in the established gesture subject list memory 
unit 73 from the gesture data base 72, and reads the 
featured amount of the gesture entered from the fea- 
tured handwriting amount memory unit 74. Then, the 
process determining unit 75 determines a process op- 
eration in response to these read data to control a 
subject line segment retrieving unit 77, a featured 
amount extracting unit 79, and a line segment proc- 
essing unit 81. 

A processed line segment list memory unit 76 
stores data about previously entered line segments 
(line segments to be processed) other than the ges- 
ture. The subject line segment retrieving unit 77 re- 
trieves the line segment to be processed from the line 
segments stored in the processed line segment list 
memory u nit 76 in response to the control by the proc- 
ess determining unit 75, and then supplies the re- 
trieved result to a subject line segment list memory 
unit 78 to be stored therein. 

A featured amount extracting unit 79 extracts the 
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featured amount of the subject line segment stored in 
the subject line segment list memory unit 78 in re- 
sponse to the control by the process determining unit 
75, and su pplies this extracted result to a featured I ine 
segment amount memory unit BO to be stored therein. 
A line segment process unit 81 process the featured 
amount of the line segment which is to be processed 
and is stored in the featured line segment amount 
memory unit 80 under control of the process deter- 
mining unit 75, and produces data used to change the 
d splay condition of the line segment. Then, the pro- 
duced data is supplied to a processed line segment 
memory unit 82 to be stored therein. 

The data stored in this processed line segment 
memory unit 82 is read out and supplied via a display 
controller 11 to a V-RAN 12, in which this process data 
is expanded as bit map data. Then, this data is read 
by the display controller 11, and then outputted to a 
display unit 13 for the representation purpose. 

Referring now to Fig. 4 to Fig. 8, such a basic 
process operation will be explained that a gesture is 
inputted in a handwriting manner by operating the pen 
1 6 on the input tablet 1 5 to establish a predetermined 
relationship with an arbitrary shape such as a pattern 
(figure) and a character which has been inputted on 
the input tablet 15 by employing the pen 16 and dis- 
played on the display unit 13, a predetermined proc- 
ess operation is performed for the previously dis- 
played shapes on the display unit 1 3, and then the dis- 
play conditions are changed. As previously stated, 
Fig. 4 to Fig. 6 represent a process operation forjudg- 
ing whether or not the inputted subject corresponds 
to the gesture, and Fig. 7 and Fig. 8 show a process 
operation to execute a suitable process operation for 
the inputted gesture. 

First of all, at a step S1 of Fig. 4, a user manipu- 
lates the pen 16 on the input tablet 1 5 to enter a single 
image. Then, the process operation defined from the 
subsequent step S2 to the step S49 is performed to 
judge whether or not a gesture is inputted. If the in- 
putted subject does not correspond to the gesture, 
then it is judged at a step S61 of Fig. 7 that no subject 
is stored in the established gesture subject list mem- 
ory unit 62. Furthermore, at a step S62, it is judged 
that no subject is either stored in the plural image ges- 
ture subject list memory unit 60, and the present proc- 
ess operation is accomplished. 

In case that a judgment is made that the inputted 
subject corresponds to a gesture command, the sub- 
ject is present in the established gesture subject list 
memory unit 62, so that a process operation defined 
after the step S65 of Fig. 7 is performed. 

To judge whether or not the inputted subject cor- 
responds to the gesture, the featured amount of the 
input handwriting entered at the step S1 is extracted 
in the featured handwriting amount extracting unit 53, 
and then stored into the input featured handwriting 
amou nt memory unit 54. The process operation is ad- 


vanced to a step S3 at which the list of the established 
gesture subject list memory unit 62 is initialized. As 
previously stated, the subject finally judged as the 
gesture (namely, subject having a higher point) is reg- 

5 istered into this established gesture subject list In ac- 
cordance with this initialization, the corresponding 
gesture number is initialized in such a manner that the 
point of this subject is substituted by, for instance, 
zero, namely the number indicative of an unprocess 

10 condition (for example, 0 or -1). 

Next, the process operation is advanced to a step 
S4 at which a judgment is made as to whether or not 
a subject is present in the plural image gesture sub- 
ject list As previously explained, when the gesture of 

15 the plural image has been inputted, the previously in- 
putted image is stored in this plural image gesture 
subject list. As a consequence, when a first subject is 
entered, the subject has not yet been registered into 
the plural image gesture subject list In this case, the 

20 process operation is advanced to a step S21 at which 
a check is done as to whether or not a single image 
gesture is located in the data base. 

When all of the gestures are arranged by the ges- 
tures of plural images, no single image gesture is in- 

25 putted . That is, in this case, the single image gesture 
data base 57 is not present in the data base. Then, in 
this case, the process operation is advanced from the 
step S21 to the step 41 , at which a check is done as 
to whether or not the plural image gesture is present 

30 in the data base, when no single image gesture data 
base 57 is present since all the gestures are con- 
structed of the plural image gestures, this plural im- 
age gesture data base 58 is located in the data base. 
Then, in this case, the process operation is advanced 

35 from the step S41 to steps S42 through S49 at which 
a check is done as to whether or not the inputted 
handwriting corresponds to the plural image gesture. 

To the contrary, when all of the gestures are the 
single image gestures, the single image gestures 

40 have been previously registered in the data base. 
Then, in this case, the process operation is advanced 
from the step S21 to steps S22 through S28, at which 
such a judging process as to whether or not the input- 
ted handwriting corresponds to the single image ges- 

45 ture. 

When the gesture is arranged by both of the sin- 
gle image gesture and the plural image gesture, there 
should be present both of the single image gesture 
data base 57 and the plural image gesture data base 
50 58. In this case, the process operation as defined 
from this step S22 to the step S28, and also the proc- 
ess operation as defined f ran the steps S42 to S49 
are executed, depending on the request 

Although it is theoretically possible to process a 
55 single image gesture as one sort of plural image ges- 
ture, the resultant judging process operation would 
become complex. Therefore, as described in the 
above embodiment it is preferable to separately proo- 
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ess the single image gesture and the plural image 
gesture. 

Returning back to Fig. 4, another process opera- 
tion is performed at a step S4 where if a judgment is 
done thatthe subject is stored in the plural image ges- 
ture subject list, namely in case of the input handwrit- 
ing subsequent to the second image, then the proc- 
ess operation is advanced to a step S5 at which the 
head subject in the plural image gesture subject list 
is selected as a first checking subject. Then, the proc- 
ess operation is advanced to a step S6 at which a pre- 
determined single image is selected as an predicted 
input subject for comparison from the plural image 
gestures of the subjects selected at the step S5. 

Next, the process operation is advanced to a step 
S7 at which a resemblance degree between the com- 
parison subject selected at the step S6 and the pre- 
sently inputted handwriting is obtained by the hand- 
writing comparing unit 56. Then, at a step S8, the 
point of the gesture subject is corrected in accor- 
dance with the resemblance degree obtained at the 
step S7. In other words, as previously stated, the re- 
semblance degrees of the respective input handwrit- 
ing among the input handwriting of the plural images 
are averaged. When this averaged value is used as 
the point of this gesture, a calculation is performed for 
both the resemblance degree of the previously en- 
tered handwriting and that of the presently entered 
handwriting, and then a newly calculated averaged 
value is set as the point of this gesture. 

This point may be calculated by employing the 
statistical value such as the above-described aver- 
age value, and also by way of the fuzzy calculation. 

Next, the process operation is advanced to a step 
S9 at which a check is done by the subject changing 
unit 61 as to whether or not the point obtained at the 
step S8 is greater than a preset reference value. 
When the point is greater than the reference value, 
the process operation is advanced to a step S11 at 
which a judgment is made whether or not the gesture 
is accomplished. In other words, the gesture of the 
plural images is now entered, when this gesture of the 
plural images is arranged by, for instance, three im- 
ages, for instance, the gesture is established if the 
third image is entered. Then, in this case, the process 
operation is advanced to a step S12at which this ges- 
ture is registered into the list of the established ges- 
ture subject list memory unit 62. 

At the step S11 , when it is judged that the gesture 
is not yet accomplished, the process operation is ad- 
vanced to a step S1 3, at which a judgment is made as 
to whether or not the subsequent gesture subject is 
present in the plural image gesture subject list (name- 
ly, it is judged whether or not there is such a gesture 
subject whose point remains initialized at the step 
S3). When the gesture of the plural images has not 
yet been established, it implies that the next gesture 
subject is still present Then, in this case, the process 


operation is advanced to a step S14 at which the 
checking subject is changed into the next gesture 
subject At the step S1 2, after the established gesture 
has been registered as the established gesture sub- 
5 ject, a similar operation is earned out when it is so 
judged at a step S13 that there is a subsequent ges- 
ture subject Then, the process operation is returned 
from the step S14 to the step S6 at which a similar 
process operation is repeated while the predicted in- 
to put image used as the checking subject at the step 
S14 is employed for comparison purposes. 

At the step S9, when it is judged that the point of 
the gesture subject is smaller than the reference val- 
ue, the process operation is advanced to a step S10 
15 at which this gesture subject is removed from the plu- 
ral image gesture subject list In other words, in this 
case, it is assumed that the presently entered hand- 
writing is not to input this gesture. 

At a step S13, when it is judged that there is no 
20 subsequent gesture subject in the plural image ges- 
ture subject list the process operation is advanced to 
a step S21 . 

As previously described, at the step S21 shown 
in Fig. 5, a judgment is made as to whether or not the 

25 single image gesture is present in the data base. 
Then, when a judgment is made of YES, the process 
operation is advanced to a step S22 at which a com- 
parison subject is set to the head gesture of the ges- 
ture stored in the single image gesture data base 57. 

30 Then, at a step S23, a calculation is earned out for a 
resemblance degree between the single image ges- 
ture of the comparison subject set at the step S22 and 
the presently inputted handwriting. At a step S24, the 
calculated resemblance degree is set as the point of 

$s this single image gesture. 

At a step S25. if the point (resemblance degree) 
of the gesture set at the step S24 is compared with a 
preset reference value and this point is greater than 
the reference value, the process operation is ad- 

40 vanced to a step S26 at which this point is registered 
into the established gesture subject list 

Either when a judgment is made that the point is 
smaller than the reference value at the step S25, or 
after the point has been registered into the establish- 
es ed gesture subject list at the step S26, the process 
operation is advanced to a step S27 at which a judg- 
ment is made as to whether or not the next gesture 
(i.e., gesture whose point is not requested) is present 
in the single image gesture data base 57. When the 

so next gesture is present the process operation is ad- 
vanced to a step S28 at which the comparison subject 
is changed into the subsequent gesture. Then, the 
process operation is returned to the step S23 at which 
a resemblance degree between the newly set gesture 

55 as the comparison subject and the input handwriting 
is calculated and a similar process operation is re- 
peated. 

At the step S27, if it is judged that there is no sub- 
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sequent gesture in the single image gesture data 
base 57, then the process operation is advanced to a 
step S41. 

At the step S41 of Fig. 6, a judgment is made as 
to whether or not the plural image gesture is present 5 
in the data base. When it is judged that the plural im- 
age gesture is present, the process operation is ad- 
vanced to a step S42. At this step S42, the checking 
subject to be compared with the input handwriting is 
set to the head gesture within the plural image ges- 10 
ture data base 58. Then, the process operation is ad- 
vanced to a step 843, at which a predetermined single 
image of the gesture recognized as the checking sub- 
ject is assumed as a predicted first image to be com- 
pared. 

Next, the process operation is advanced to a step 
S44 at which a resemblance degree between the in- 
put handwriting and the comparison subject set at the 
step S43 is calculated. Then, the calculated resem- 
blance degree is understood as the point of the ges- 
ture at a step S45. At a step S46, a judgment is made 
as to whether or not the point (resemblance degree) 
of the gesture obtained at the step S45 is larger than 
the reference value. When it is judged that the point 
of the gesture is greater than the reference value, this 
point of the gesture is stored into the plural image 
gesture subject list memory 60 as the plural image 
gesture subject 

Eitherwhen it is so judged atthe step S46 that the 
resemblance degree is smaller than the reference val- 
ue, a- when the gesture has been completely regis- 
tered into the plural image gesture subject list at the 
step S47, the process operation is advanced to a step 
S48. At this step S48, a check is done as to whether 
or not the subsequent gesture (namely, gesture 
whose point is not yet updated) is located in the plural 
image gesture data base 58. When there is the next 
gesture, the process operation is advanced to a step 
S49 at which the checking subject is set to the next 
gesture. Then, the process operation is returned to 
the step S43 at which a first predicted image of this 
checking subject is compared with the input handwrit- 
ing, and this comparing process is repeatedly per- 
formed in a similar manner to the above case. 

When it is so judged at the step S48 that no next 
gesture is present in the plural image gesture data 
base 58, the process operation is advanced to a step 
S61 shown in Fig. 7. 

With the above-described process operations, a 
judgment is made as to whether or not the inputted 
subject corresponds to the gesture. If the inputted 
subject corresponds to the gesture, then this gesture 
is described as a subject in the established gesture 
subject list In this case, the process operation is ad- 
vanced to a step S65 at which the subsequent proc- 
ess operation is carried out 

If the inputted subject does not correspond to the 
gesture, then there is no subject tin the established 


gesture subject list Then, in this case, the process 
operation is advanced to the step S62 at which a judg- 
ment is made as to whether or not the gesture subject 
of the plural images is present If the inputted subject 
does not correspond to the gesture, then there is no 
data also in the plural image gesture subject list In 
this case, the process operation is accordingly ended. 

On the other hand, it is so judged at the step S61 
that no subject is present in the established gesture 
subject list in a halfway when the plural image ges- 
ture is entered. It should be noted that in this case, a 
judgment is made at the step S62 that there is a sub- 
ject in the plural image gesture subject list Then, the 
process operation is advanced to a step S63, at which 
the input handwriting is added to the plural image 
handwriting in the plural image gesture subject list 
memory unit 60. Thereafter, the process operation is 
advanced to a step S64 at which the input featured 
amount is added to the plural image featured amount 
in the plural image featured handwriting amount 
memory unit 55. Then, the process operation is again 
returned to the step S1 at which a similar process op- 
eration is carried out for the newly entered input hand- 
writing, as previously explained. 

At steps succeeded to the step S65, a command 
corresponding to an inputted gesture is executed. 
That is, at a step S65, a selection is made of a subject 
having the highest point from the subjects stored in 
the established gesture subject list memory unit 62. 
Subsequently, the process operation is advanced to 
a step S66 at which a detection is made of such a fea- 
tured amount for the handwriting of the gesture re- 
quired to change the data about the line segment 
which has been inputted and displayed on the display 
unit 13. This necessary featured amount may be ob- 
tained from the gesture date base 72. 

Next, the process operation is advanced to a step 
S67 at which the list of the line segment to be proc- 
essed by the gesture is initialized. Then, the process 
operation is advanced to a step S68, at which the fea- 
tured amount (parameter) of the gesture obtained at 
the step S66 is utilized to retrieve the line segment to 
be processed. The retrieving method for the process- 
ing subject has been stored in the gesture data base 
7Z 

Next, at a step S69, as a result of the retrieving 
operation at the previous step S66, a judgment is 
made as to whether or not the line segment to be proc- 
essed is retrieved. When no line segment to be proc- 
essed is retrieved, the process operation is advanced 
to a step S70. At this step, another judgment is made 
as to whether or not another subject is present in the 
established gesture subject list When another sub- 
ject is present, the process operation is advanced to 
a step S71 at which a subject having the second high- 
est point is selected. Then, the process operation is 
returned to the step S66, at which the featured 
amount required to execute the command of this se- 
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lected subject is extracted, and the subsequent proc- 
ess operation is similarly repeated. 

At a step S70, when it is so judged that there is 
not another subject in the established gesture subject 
list, the process operation is returned to the step S62 
at which a check is done as to whether or not a subject 
is present in the plural image gesture subject list 
Then, the subsequent process operations are si milar- 
ly executed in response to this judgment result. 

The reason why the process operation is re- 
turned from the step S70 to the step S62 and the sub- 
sequent process operation is repeated, is to confirm 
whether or not it is recognizable that the plural image 
gesture is being entered. 

At the step S69, when it is so judged that the line 
segment to be processed for the gesture is retrieved, 
the process operation is advanced to a step S61 rep- 
resented in Fig. 8. 

At this step S81 , a confirmation is made of a fear 
tu red amount (parameter) of a line segment which 
should be processed. The necessary featured 
amount can be recognized from the gesture data 
base 72. At a step S82, the line segment data after 
process is calculated by utilizing the process content 
stored in the gesture data base 72, the featured 
amount of the gesture, and the featured amount of the 
line segment to be processed. Then, the calculated 
data is stored in a processed line segment memory 
unit 82. 

Subsequently, the process operation is ad- 
vanced to a step S83, at which a confirmation is made 
as to whether or not the process operation corre- 
sponding to the gesture is executable. For instance, 
such a confirmation whether or not the process oper- 
ation is executable is given to a user by representing 
a change made when the gesture command is exe- 
cuted by varying its colour. As a concrete confirma- 
tion method, a region corresponding to YES/NO is 
displayed on the display unit 13, and any of YES/NO 
is touched by a user with the pen 16. When the user 
refuses this execution, the process operation is re- 
turned to the step S70 at which a confirmation proc- 
ess for another gesture is performed. 

When the user allows the execution of the ges- 
ture, the process operation is advanced to a step S84 
at which the handwriting of the gesture is erased. For 
example, when such a handwriting having no relation- 
ship with the gesture is allowed to be entered during 
the input operation of the plural image gesture by 
adding such a process operation that a constructive 
handwriting number is added to the gesture subject, 
only the handwriting for constituting the confirmed 
gesture command may be erased. 

At the next step S85, the line segment data is 
changed into the data calculated at the step S82. 
Concretely speaking, the line segment data to be 
processed which has been stored in the subject line 
segment list memory unit 78 is removed from the 


processed line segment list memory unit 76, and then 
the line segment data after being changed, which has 
been stored in the processed line segment memory 
unit 82 is added. As a consequence, the previously 
5 entered and displayed line segment is changed into 
such a display state corresponding to the entered 
gesture. 

Next, the list of the plural image gesture subject 
list memory unit 60, the plural image handwriting data 
10 of the plural image handwriting memory unit 52, and 
the data about the plural image featured handwriting 
amount of the plural image featured handwriting 
amount memory unit 55 are initialized at steps S86 to 
S88. 

15 As described in the above case, when the hand- 
writing having no relationship is allowed to be dis- 
played during the input operation of the plural image 
gesture by adding the handwriting number for consti- 
tuting the gesture subject, only the subject containing 

20 the handwriting for constituting the fixed gesture 
command is removed. In this case, the operation 
mode may be escaped from the gesture input mode 
by preparing such a gesture command which can in- 
validate all of the plural image gestures under entry. 

25 After the process operation of the step S88 
shown in Fig. 8, when the gesture input operation is 
complete, the operation mode may be automatically 
changed from the gesture command input mode into 
the pattern input mode. 

so The above-described process operations repre- 
sented in the flow charts from Fig. 4 to Fig. 8 are a 
basic process operation. These process operations 
are slightly different in accordance with the concrete 
gestures. A description will now be made of this dif- 

35 ferent portion. 

Fig. 9 to Fig. 13 represent a display example of 
gestures in case that line segments which have been 
previously entered and displayed are aligned in a 
equidistant interval. To perform such a process oper- 

40 ation, the process operation of the step S66 is exe- 
cuted as shown in Fig. 14, the process operation of 
the step S68 carried out as indicated in Fig. 15, the 
process operation of the step S81 is performed as 
shown in Fig. 16, and the process operation of the 

4$ step S82 is performed as denoted in Fig. 1 7. 

As illustrated in Fig. 9, in case of this embodi- 
ment, three straight lines L i( U, U are displayed in 
such a manner that these straight line are mutually 
extended along the substantially upper and lower di- 

so recttons are located parallel to each other with differ- 
ent intervals. A straight line U 1 is displayed on the left 
side of the straight line Li undersubstantially horizon- 
tal condition in such a manner that a predetermined 
angle is defined by the straight line U and other 

55 straight lines U and U Similarly, another straight line 
L 12 is displayed on the right side of the straig ht line L 3 
under substantially horizontal condition in such a 
manner that a certain angle is defined between this 
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straight line L 12 and the straight line La- 
Assuming now that a user changes the display 
condition of the three straight lines L 1( U, U into an 
equidistant interval display condition. In this case, as 
illustrated in Fig. 9, the user inputs such a gesture Si 
that two half circular patterns (figures) are continu- 
ously connected to each other by way of a pen 16. A 
starting point of the gesture Si is Q 1§ and an end point 
thereof is Q ie . 

The process operation of the step S66 corre- 
sponds to a process operation for extracting a fear 
tured handwriting amount of a gesture. In this case, 
as shown in Fig. 14, coordinate values of the starting 
point Q 1t and the end point Qi e of the gesture (input 
handwriting) &i are acquired as the featured 
amounts. Then, at a step S1Q2, a straight line Ui con- 
taining the starting point Qi 8 and the end point Q 1a is 
extracted as represented in Fig. 10. 

On the other hand, the step S68 of Fig. 7 corre- 
sponds to a process operation for retrieving a line 
segment which corresponds to a subject to process a 
gesture command. This process operation is per- 
formed as shown in Fig. 1 5. In other words, at a first 
step S1 11 , a process subject is set to a head line seg- 
ment Li of the list in the processed line segment list 
memory unit 76. For example, it is now assumed that 
LfL,. Thereafter, the process operation is advanced 
to a step S112 at which a judgment is made as to 
whether or not the data about the line segment Li set 
at the step S111 is present 

When this data is present, the process operation 
is advanced to a step S113 at which a check is done 
as to whether or not this line segment Li intersects 
with the gesture Si. If the line segment Li intersects 
with the gesture Si, the process operation is ad- 
vanced to a step S114 at which a judgment is made 
whetheror not an angle defined between the line seg- 
ment Li and the straight line Ui is greater than, or 
equal to a preset reference value. In other words, 
when the angle defined between the line segment Li 
and the gesture S 1 is smaller than the reference val- 
ue, this line segment Li is excluded from the process 
subject. When the angle between both of the line seg- 
ment Li and the gesture Si is greater than, or equal 
to the reference value, the process operation is ad- 
vanced to a step S11 5 at wh ich this line segment Li is 
registered to the list of the processed line segment list 
memory unit 76. 

Subsequently, the process operation is ad- 
vanced to a step S116 at which a variable Y is incre- 
mented by 1, and the next line segment in the proc- 
essed line segment list is used to be processed. At the 
step S113, when it is so judged that the line segment 
Li intersects with the gesture Si , and also at the step 
S114, when the angle defined between the line seg- 
ment Li and the straight line Ui is below than the ref- 
erence value, the process operation is advanced to a 
step S116 at which the subject is moved to the next 


line segment 

In case of this embodiment, as shown in Fig. 11, 
since the three straight lines L< to L a intersect with the 
gesture Si at a sufficiently large angle, it is judged 

5 that these three straight lines L t to U are to be proc- 
essed, and these straight lines are stored into the 
subject line segment list 

On the other hand, the step S81 shown in Fig. 8 
corresponds to a process operation for confirming a 

10 featured amount of a subject line segment, and this 
process operation is carried out as shown in Fig. 16 
in this embodiment. First, at this step S121 , the head 
line segment Li of the subject line segment list is first 
processed. Next, the process operation is advanced 

is to a step S122 at which there is data about the line 
segment Li. If the data about the line segment U is 
present then the process operation is advanced to a 
step S123, at which a cross point Pi between the line 
segment Li and the straight line Ui is obtained. 

20 Subsequently, the process operation is ad- 
vanced to a step S124 at which a distance "Di" be- 
tween the cross point Pi and the starting point Qi» of 
the gesture Si is calculated. Then, the process oper- 
ation is advanced to a step S1 25 at which the variable 

25 T is incremented by 1 , and a line segment to be proc- 
essed is changed into the next line segment. Then, 
the process operation is returned to the step S122 at 
which a similar process operation is repeated. 
At the step S122, when it is judged that the data 

30 about the line segment Li is not present, the process 
operation is not present, the process operation is ad- 
vanced to a step S62. 

In case of this embodiment, as illustrated in Fig. 
12, a cross point Pi between the straight line h and 

35 the straight line Ui. another cross point P 2 between 
the straight line l 2 and the straight line L»i. and an- 
other cross point P 3 between the straight line U and 
the straight line L a1 are obtained. Also, distances D 1p 
D2, D 3 from the starting point Q 18 of the gesture Si up 

40 to the cross points P 1p P 2 . Ps are obtained. 

A process operation defined at the step S62 of 
Fig. 8 corresponds to a process operation for per- 
forming a calculation of data about a processed line 
segment. This process operation is performed as 

4$ shown in Fig. 1 7 in case of this embodiment In other 
words, at a first step S131 , a variable N indicative of 
the number of line segments registered in the subject 
line segment list is set to "0". Next, at a step S1 32, the 
head line segment Li of he subject line segment list is 

so to be processed. 

Next, the process operation is advanced to a step 
S1 33 at which a judgment is made as to whether or 
not the data about the line segment Li is present If 
this data about the line segment is present then the 

55 process operation is advanced to a step S134 at 
which the variable N is incremented by 1. Then, the 
process operation is advanced to a step S135 at 
which the variable V is incremented by 1, and also is 
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returned to the step S133 at which a similar process 
operation is repeated. That is, the quantity of existing 
line segments is counted by this process operation, 
and is set to the variable N. 

When it is so judged at the step S1 33 that no data 
aboutthe line segment Li is present (namely, all of line 
segments in list are counted), the process operation 
is advanced to a step S136, at which the subject line 
segment list is sorted in such a manner that the line 
segment data are sequentially aligned in the order of 
small distances DL Subsequently, at a step S1 37, the 
head line segment Li of the processed line segment 
1st is set to a subject to be processed. Then, the proc- 
ess operation is advanced to a step S138 at which a 
check is done as to whether or not the data on this line 
segment Li is present If there is such a data aboutthe 
line segment Li, then the process operation is ad- 
vanced to a step S139. At this step, the cross point Pi 
between the line segment Li and the line segment Ui 
is moved to such a point determined by dividing the 
d stance between the starting point Q u and the end 
point Qi e of the gesture Si at a ratio of (H) to (M-i). 
Thereafter, the process operation is advanced to a 
step S 140 at which after the variable V is increment- 
ed by 1 , the process operation is returned to the step 
S1 36 where a similar process operation is repeated. 

When it is so judged at the step S1 38 that there 
is no data about the line segment Li, the process op- 
eration is transferred to the process operation de- 
fined at the step S63. 

In case of the present embodiment, as shown in 
Fig. 13, since the subject line segments are L 1( L 2l L 3 , 
it is set N=3. Then, the cross point Pi between the 
straight line Li and the straight line Ui is arranged to 
a position coincident with the starting point Q u of the 
gesture S 1t and the cross; point P$ between the 
straight line U and the straight line Ui is moved to a 
position coincident with the end point Q 1fl of the ges- 
ture Si. Then, the cross point P2 between the straight 
line U and the straight line Ui is moved to a midpoint 
between the end point Q 1e and the starting point Q lB 
of the gesture Si. As a result, the straight lines Li to 
L 3 are moved and displayed in such a manner thatthe 
interval between the straight line Li and the straight 
line U is made equal to the interval between the 
straight line l 2 and the straight line L*. 

In an embodiment shown in Fig. 18 to Fig. 22, 
such a process operation is carried out that two 
straight lines arranged at a predetermined angle are 
located at a right angle, namely a trailing process is 
performed. In this case, the respective process oper- 
ations defined at the steps S66, S68, S81, or S82 
among the process operations shown in Fig. 4 to Fig. 
8 are performed as illustrated in Fig. 23, Fig. 24, Fig. 
25, or Fig. 26. 

To perform the process operation for extracting 
the featured amount of the gesture handwriting de- 
fined at the step S66, coordinate values of a starting 


point Qfe and an end point Q^ofa gesture S2 are de- 
tected at a step S151 in this embodiment, as illustrat- 
ed in Fig. 23. 

In other words, as shown in Fig. 18 and Fig. 19, 

5 in accordance with this embodiment the gesture S2 
constructed of a line segment bent at a substantially 
90° is written in such a manner that the starting point 
thereof is and the end point thereof is Q25 in order 
to establish such a relationship that the straight lines 

10 L 3 and L 12 are positioned at a right angle. Both the 
starting point and the end point are detected. 

At the step S68 of Fig. 7, the line segment to be 
processed is retrieved, so that this process operation 
is carried out as illustrated in Fig. 24. That is, at a first 

15 step S161, the head line segment Li of the processed 
line segment list is first processed. Then, at a step 
S162, a judgment is made as to whether or not there 
is the data about the line segment Li. If this data is 
present then the process operation is advanced to a 

20 step S1 63 at which a judgment is made as to whether 
or not a distance between the line segment Li and the 
gesture $2 is smaller than, or equal to a preset refer- 
ence value. When it is so judged the distance is small- 
er than, or equal to the reference value, the process 

25 operation is advanced to a step S164 at which the line 
segment Li is registered into the subject line segment 
list 

When it is so judged at the step S163 that the dis- 
tance between the line segment Li and the gesture S2 

30 greatly exceeds the reference value, this line seg- 
ment Li is not added to the subject line segment list, 
but removed from the process subject That is, in this 
case, such a line segment whose the distance be- 
tween the gesture S2 and the line segment is smaller 

35 than the reference value, is to be processed. 

After the judgment process has been complete 
for the line segment Li, the process operation is ad- 
vanced to a step S165 at which the variable T is in- 
cremented by 1, and the process operation is re- 

40 turned to the step S162 at which a similar process op- 
eration is repeated. 

At the step S162, when it is so judged that there 
is no date about the line segment Li, the process op- 
eration is advanced to the step S68. 

45 As shown in Fig. 19, in case of this embodiment 
since the straight line present at the distance below 
than the reference value for the gesture S2 corre- 
sponds to the straight lines U and L 12 , these two 
straight lines are to be processed. 

so Next, to perform the process operation for con- 
firming the featured amount of the line segment to be 
processed at the step S81 of Fig. 8, a process oper- 
ation shown in Fig. 25 is performed. That is, in this 
embodiment the head line segment Li of the subject 

55 line segment list is first processed at the step S171. 
Then, the process operation is advanced to a step 
S172 at which a check is done whether or not there 
is data about this line segment Li. When the data on 
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the line segment Li is present the process operation 
is advanced to a step S1 73 at which such a point P b 
whose distance between the stating point of the 
gesture S2 and itself is minimum is obtained. 

Subsequently, at a step S174, a distance be- 
tween the point P ta obtained at the step S1 73 and the 
starting point Q21 of the gesture S3 is assumed as a 
distance D4 between the line segment Li and the 
starting point Q2,. 

Similarly, at a step S175, a point P to is obtained 
whose distance from the end point of the gesture 
S 2 on the line segment Li. At the next step S176, the 
distance between the point P te and the end point 
is set as a distance Dei between the line segment Li 
and the end point Q^. 

At a step S1 77, a direction "ai" of the line segment 
Li is calculated. Then, a step S178, after the variable 
T is incremented by 1, the process operation is re- 
turned to the step S1 72 at which a similar process op- 
eration is repeatedly performed. When it is so judged 
at the step S172 that there is no data about the line 
segment, the process operation is advanced to a step 
SB2. 

The calculation of the line segment data after be- 
ing processed at the step S82 of Fig. 8 is performed 
as defined from the step S1 81 to S1 94 of Fig. 26. That 
is, at a step S1 81 , the head line segment Li in the sub- 
ject line segment list is first processed. Then, a judg- 
ment is made at a step S1 82 as to whether or not the 
data on the line segment Li is present If there is this 
data, then the process operation is advanced to a step 
S183. At this step S183, a straight line whose dis- 
tance "D/ from the starting point of the gesture 
S 2 is minimum is set as Ls, a direction "ai" of this 
straight line is set as a., and a point "Pi." on this 
straight line, whose distance is the shortest from the 
starting point Q2,. 

In other words, at the step S183, a distance D«n 
of a line segment U 1( which has been processed just 
before, is compared with the distance D c) of the line 
segment Li, which is presently process, so that either 
the straight line having the shorter distance is set as 
Ls. As a result, the straight line which is located at the 
nearest position from the starting point Q2, of the ges- 
ture S2, is selected as a first processing line segment 
Ls. 

Next, the process operation is advanced to a step 
S1 84 at which a straight line whose distance D d from 
the end point of the gesture S 2 is minimum Is set 
as Le, the direction ai of this straight line is set to a e , 
and the point P to on this straight line, which is located 
at the nearest position from the end point of the 
gesture S2 is set to a point P e . That is, the distance 

1 of the previously processed line segment L|_ 1 is 
compared with the presently processed distance D e[ , 
and then either the straight line having the shorter dis- 
tance is set as Le. As a result, the straight line which 
is located at the nearest position from the end point 


Q&, of the gesture S2 is selected as a second process- 
ing line segment Le. 

Thereafter, the process operation is advanced to 
a step S185 at which the variable T is incremented 
5 by 1. Then, the process operation is returned to the 
step S182 at which a similar process operation is re- 
peated. 

At the step S182, when it is so judged that there 
is no data about the line segment Li (retrieving oper- 

10 ations for data on all line segments Li have been ac- 
complished), the process operation is advanced to a 
step S186 at which a comparison is made between 
the direction as of the line segment Ls located at the 
nearest position from the starting point of the ges- 

15 ture S2 and the direction ae of the line segment Le lo- 
cated at the nearest position from the end position 
Qte. When either as is larger than, or equal to as, the 
process operation is advanced to a step S187 at 
which a judgment is made as to whether or not a value 

20 obtained by subtracting a, from a, is larger than 90°. 
When (ae-a^ is larger than 90°, the process op- 
eration is advanced to a step S188 at which a value 
of 180°-(a a -aJ is set to "angle". When the value of {a*- 
a.) is smaller than, or equal to 90°, the process oper- 

25 ation is advanced to a step S189 at which the value 
of (ai-a.) is directly set to "angle". In other words, an 
acute angle (i.e., smaller angle} among the angles de- 
fined between the line segment Ls and the line seg- 
ment Le is set to "angle". 

so Then, the process operation is advanced from 
either the step S188 or the step S189 to a step S190 
at which the line segment Ls located at the nearest 
position from the starting position of the gesture 
S 2 is rotated by an angle {(a^-a^-OO 0 ) around the point 

35 Ps located at the nearest position from the starting 
point Qai of the gesture S2 as a centre. As a result, the 
line segment Ls close to the starting point of the 
gesture S 2 is rotated by a preselected angle with re- 
spect to the line segment Le located at he nearest 

40 position from the end point Q^, whereby the angle 
between the line segment Ls and the line segment Le 
becomes 90°. 

When it is so judged at the step S1 86 that the an- 
gle a,, is greater than the angle a e , the process oper- 

4$ ation is advanced to a step S191 at which another 
judgment is made whether or not (a^aj is greater 
than 90°. When (a^) is greater than 90°, the proo- 
ess operation is advanced to a step S92 at which a 
value of 180°-(a»-ae) is set to "angle". When It is so 

so judged that the value of (a^) is smaller than, or 
equal to 90°, the process operation is advanced to a 
step S193 at which the value of (a B -aJ is set to "an- 
gle". 

In other words, similar to the above-described 
55 case, the smaller angle (acute angle) is set to "angle" 
among the angles defined between the two straight 
lines Ls and Le. Then, at a step S194, the straight Ls 
is rotated by the angle of 90°-(a a -a*) around the point 
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Ps as a centre, and is inclined by 90° with respect to 
the straight line Le. 

In the embodiment shown in Fig. 18 to Fig. 22, 
since the straight line located at the neatest position 
from the starting point of the gesture S2 is U, and 
the straight line positioned at the nearest location 
from the end point is L 12 , these two straight lines 
L s and L,2 are selected as the processing subject In 
Fig. 21 , the point Pn is located on the straight line U. 
at the nearest position from the starting point of 
the gesture S 2 , whereas the point P 12 is located on the 
straight line L 12 , at the nearest position from the end 
point Q2 6 of the gesture S* Also, 03 represents the di- 
rection (angle) of the straight line L 3 and a 12 indicates 
the direction (angle) of the straight line L 12 . 

Although the angle has been set along the coun- 
ter clockwise direction with respect to the horizontal 
direction in the above-described embodiment, this 
angle may be set along the clockwise direction. Then, 
in this embodiment, the angle is normalized within a 
range defined from the positive direction (0°) of the 
X axis to the negative direction (180°) of the X axis. 

I n Fig. 22, there is shown such a condition that the 
straight line U has been rotated by 90° with respect 
to the straight line L 12 . 

In the above-explained embodiment, the straight 
line located at the nearest position from the starting 
point of the gesture is rotated. Alternatively, either the 
straight line located dose to the end point of the ges- 
ture, or both of these straight lines may be rotated. 

Fig. 27 through Fig. 31 schematically represent 
such an embodiment that edge points of line seg- 
ments are aligned on a predetermined straight line. 
When such an embodiment is realized, the process 
operation defined at the step S68 of Fig. 7 is executed 
by a process operation defined at a step S201 of Fig. 
32. That is, as represented in Fig. 27, when a gesture 
S 3 is inputted in which one end portion of a sufficient- 
ly long straight line is bent at a short portion thereof, 
a straight line L involving the longer line segment in 
this gesture S3 is detected (see Fig. 28). 

It should be noted that this gesture S 3 has such 
a shape having a bending angle at a substantially 90°, 
and when one portion of this gesture S3 owns a twice 
longer length than the reference value, as compared 
with the other portion, this gesture S fl is recognized 
as a gesture whose end points are aligned. 

Then, the process operation to retrieve the line 
segment to be processed as defined at the step S68 
of Fig. 7 is carried out shown in Fig. 33. That is, the 
head line segment Li of the processed line segment 
list is selected as a first subject to be processed at a 
step S211. Thereafter, a judgment is made at a step 

5212 as to whether or not there is data about the line 
segment Li. When the data about the line segment is 
present, the process operation is advanced to a step 

5213 at which another judgment is made as to wheth- 
er or not a distance between the gesture S3 and one 


end point P to of the line segment Li is smaller than, or 
equal to a preset reference value. When the end point 
Pit is positioned close to the gesture Ss, the process 
operation is advanced to a step S21 5 at which this line 
5 segment Li is registered into the subject line segment 
list 

At a step S213, when it is so judged that the edge 
point P| ft is relatively largely separated from the ges- 
ture S 3> the process operation is advanced to a step 

10 S214. At this step S214, another judgment is made as 
to whether or not a distance between the gesture S3 
and the other edge point P lb of this line segment Li is 
shorter than the reference value. When this distance 
is shorter than the reference value, the process oper- 

15 ation is advanced to a step S215 at which this line 
segment Li is registered into the subject line segment 
list 

Subsequently, the process operation is ad- 
vanced to a step S216 at which the variable T is in- 

20 cremented by 1, and the process operation is re- 
turned to the step S212 at which a similar operation 
is repeatedly performed. Thus, the line segments of 
the gesture S 3 to be processed are retrieved by re- 
peating the above-described process operation. 

25 In case of the embodiment shown in Fig. 27 to 
Fig. 31, there are straight lines U, U. U as the 
straight lines whose edge points are arranged near 
the gesture S3. Asa result, these three straight lines 
are selected as the process subject by the gesture S3 

so in this case. 

Next, the process operation to confirm the fea- 
tured amount of the subject line segment as defined 
at the step S81 in Fig. 8 is carried out as illustrated 
from a step S221 to S228 of Fig. 34. That is, at a first 

35 stepS221 , the head line segment Li of the subject line 
segment list is first processed, and at a step S222, a 
confirmation is made as to whether or not there is the 
data about this line segment Li. When the data about 
the line segment Li is present, the process operation 

40 is advanced to a step S223. At th is step, a calculation 
is made of a distance D ta between the straight line L 
corresponding to the gesture S3 confirmed at this 
step S201 of Fig. 32 and one edge portion P !a of the 
line segment Li. Similarly, another distance Do, be- 

45 tween this straight line L and the other edge point Pq> 
is calculated at a step S224. 

At a next step S225, the dimension of the dis- 
tance 0^ is compared with that of the distance D| b . 
The edge point P tof or Po> corresponding to the dis- 

50 tance having the smaller dimension is set to a point 
Pi at either a step S226, or a step S227. 

Subsequently, at a step S228, the variable V is 
incremented by 1 , and then the process operation is 
returned to a step S222 at which a similar process op- 

55 eration is repeatedly performed. With this manner, 
the edge point located at the nearest position from the 
straight line L corresponding to the gesture S 3( among 
the edge points of these line segments, is stored as 
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Pi. 

In the embodiment shown in Fig. 30, the edge 
points P 2 i, P22. P23 of the straight lines L 1f U, U, as 
indicated in the upper portion of this drawing, are stor- 
ed as the edge points located at the nearest point 
from the straight line L corresponding to the gesture 
S* 

The process operation to calculate the processed 
line segment data as defined at the step S82 of Fig. 
6 is executed as illustrated in a step S231 to a step 
S234 of Fig. 35. In other words, at the first step S231 , 
a selection is made of the head line segment U of the 
subject line segment list At a step S232, a confirma- 
tion is made whether or not the data about this line 
segment Li is present When the data about the line 
segment U is present the edge point Pi located near 
the straight line L corresponding to the gesture S$ of 
this line segment is moved on the straight line L 
Then, the process operation is advanced to a step 
S234 at which the variable "i" is incremented by 1. 
Then, the process operation is returned to the step 
S232 at which a similar process operation is repeat- 
ed. Thus, the edge point of the line segment to be 
processed is moved on the straight line L. 

In the embodiment shown in Fig. 31, the edge 
points P21 to P23 of the straight lines Li to U are 
moved on the straight line L 

It should be noted that although the straight lines 
Li to L3 are moved in parallel to this direction in this 
embodiment, these straight lines may be moved in 
parallel to the X-axis direction, the Y-axis direction, or 
the direction perpendicular to the straight line L Fur- 
thermore, the edge points P 2i to P23 are moved on the 
straight line L, but may be moved in such a manner 
that these edge points are moved in parallel to the 
straight line L and on the straight line in such a way 
that a summation of the distances among these edge 
points P 2 i to P 2 s become minimum. 

Alternatively, these edge points may be moved in 
parallel to the straight line Land on the straight line 
in such a manner that a powered summation of the 
distances among these edge points P21 to P23 be- 
comes minimum. 

Fig. 36 to Fig. 40 show such an embodiment that 
arbitrary straight lines are aligned in parallel to each 
other at a predetermined angle. To realize such a 
process operation, the process operations defined at 
the step S66 and the step S68 shown in Fig. 7, and 
the steps S82 and S82 indicated in Fig. 8 are execut- 
ed as illustrated in Fig. 41 to Fig. 4. 

In other words, as shown in Fig. 41, the detection 
of the featured amount of the gesture is carried out as 
follows: In case of this embodiment, the gesture is ar- 
ranged by 3 images. As illustrated in Fig. 36, as the 
first gesture, a gesture S 4 is drawn in the form of a sin- 
gle straight line in such a manner that this straight line 
intersects with line segments U to U which should be 
processed. 


As shown in Fig. 3, thereafter, gestures S 5 and Sa 
whose lengths are sufficiently shorter than that of the 
gesture S 4 are entered in such a manner that these 
gesture S 5 and S a intersect with the gesture S 4 at an 

5 angle larger than a predetermined reference value. At 
a step S241, as illustrated in Fig. 36, a straight line 
L,4 corresponding to the gesture S 4 is extracted. At 
the next step S242, another straight line Us corre- 
sponding to the gesture S 5 is extracted. Furthermore, 

10 at a step S243, a direction A5 of the straight line L* 
is detected. 

As the process operation corresponding to the 
step S68, as shown in Fig. 42, the head line segment 
Li of the processed line segment list is first selected 

15 as the subject to be processed at a step S2S1 . At a 
step S252, a judgment is made as to whether or not 
the data about this line segment Li is present. When 
it is so judged that the data on the line segment Li is 
present the process operation is advanced to a step 

20 S253. At this step S253, another check is made as to 
whether or not does the line segment Li intersect with 
the gesture S 4 . When it is so judged that the line seg- 
ment Li intersects with the gesture S 4 , the process 
operation is advanced to a step S254 at which a judg- 

25 ment is made as to whether or not an intersecting an- 
gle between the line segment Li and the gesture S 4 
is larger than, or equal to the reference value. If this 
intersecting angle is larger than, or equal to the refer- 
ence value, then the operation is advanced to a step 

30 S255 at which this line segment Li is registered into 
the subject line segment list 

When it is so judged at the step S253 that the line 
segment Li does not intersects with the gesture S 4p 
when it is so judged at the step S254 that the angle 

35 between the line segment Li and the straight line U* 
is below than the reference value, and when the reg- 
ister process operation for the processed line seg- 
ment list has been complete at the step S255, the 
process operation is advanced to a step S256 at 

40 which the variable Y is incremented by 1, and then 
the process operation is returned to the step S252. 
Then, a similar process operation is repeated. 

With the above-described process operation, line 
segments designated by the gesture S 4 and the ges- 

4$ ture S 5 as the process subject are retrieved. In the 
embodiment of Fig. 39, three straight lines U, U, U 
are registered into the subject line segment list as the 
subject to be processed. 

As the process operation defined at the step S81 

so of Fig. 8, a process operation as defined from a step 
S261 to a step S265 shown in Fig. 43 is carried out 
In other words, at a first step S261 , the head line seg- 
ment Li in the subject line segment list is selected as 
the subject to be processed, and then, a judgment is 

55 made at a step S262 as to whether or not the data 
about this line segment Li is present When it is so 
judged at the step S262 whether or not the data about 
the line segment Li is present the process operation 
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is advanced to a step S263 at which a cross point Pi 
between the line segment Li and the gesture S* is ob- 
tained. Then, a direction ai of the I ine segment Li is ob- 
tained at a step S284. Then, the process operation is 
advanced to a step S265 at which the variable T is 
incremented by 1 , and then the process operation is 
returned to the step S262 at which a similar process 
operation is repeated. 

In case of the above-described embodiments 
shown in Fig. 38 and Fig. 39, the cross points P31. Pa* 
Pas between the line segments L, , L 2 , L 3 and the ges- 
ture S 4 are obtained, and also the directions a 1( 83 
of the respective line segments are obtained with the 
abovedescribed process operations. 

Furthermore, the calculation for the processed 
line segment data at the step S82 of Fig. 8 is carried 
out as defined from a step S271 to a step S274 of Fig. 
44. At a first step 271 , the head line segment Li of the 
processed line segment list is selected as the subject 
to be processed. At a step S272, a judgment is made 
as to whether or not there is data about this line seg- 
ment Li. When it is judged that the data about the line 
segment Li is present, the process operation is ad- 
vanced to a step S273 at which this line segment Li 
is rotated by an angle (As-ai) around a cross point Pi 
between the line segment Li and the gesture S 4f 
which Is obtained at the step S263 of Fig .43, as a cen- 
tre. In other words, the line segment Li is rotated in 
such a manner that the angle becomes As (namely be- 
comes in parallel to the gesture S 5 ). 

Next, the process operation is advanced to a step 
S274 at which the variable V is incremented by 1. 
Thereafter, the process operation is returned to the 
step S272 at which a similar process operation is re- 
peated. 

As illustrated in Fig. 40, the line segments L 1( U. 
L 3 to be processed are rotated around the cross 
points P31. P32. P33 between these line segments and 
the gesture S 5 in such a manner that these line seg- 
ments Li, L 2 , U are positioned in parallel to the 
straight line Us corresponding to the gesture S5. 

It should be noted that although in the above- 
described embodiment, the straight lines It to L 3 have 
been corrected in such a manner that they are equal 
to the angle A5 corresponding to the straight line Us, 
they may be equal to either an angle corresponding 
to an average value between the angle A* defined by 
the gesture S e and the angle As defined by the ges- 
ture S5, or an angle corresponding to an average val- 
ue among the angles ai to a 3 of the straight lines L, 
toU 

Fig. 45 to Fig. 48 represent such an embodiment 
that edge points of both ends of a line segment are 
aligned on a preselected line. In this case, the process 
operations as defined in the steps S66, S68, S81 , S82 
shown in Fig. 7 or Fig. 8 are carried out as illustrated 
in Fig. 49 to Fig. 52. 

That is, in accordance with the present embodi- 


ment, as illustrated in Fig. 45, a gesture is arranged 
by a gesture S 7 which is linearly drawn adjacent to 
one edge points of the line segments U, U. U to be 
processed, and a gesture S e which is linearly drawn 

5 adjacent to the other edge points thereof. 

AtastepS281 andastepS282ofFig. 49, straight 
lines U? and Us corresponding to these two gesture 
S 7 and gesture S e are extracted as shown in Fig. 46. 
Next, as illustrated in Fig. 50, the process oper- 

10 ations as defined from a step S291 to a step S298 are 
earned out, thereby to retrieve a line segment to be 
processed. In other words, at a first step S291 , a se- 
lection is made of the head line segment Li in the proc- 
essed line segment list as the processing segment. At 

1$ a step S292, a check is done as to whether or not 
there is data about this line segment Li When it is so 
judged that the data about theline segment Li is pres- 
ent, the process operation is advanced to a step S293 
at which another judgment is made as to whether or 

20 not a distance between the straight line U? and one 
edge point P lQ of the straight line Li is smaller than or 
equal to a preset reference value. 

When the distance between the edge point P^ 
and the straight line U7 is smaller than the reference 

25 value, a check is done as to whether or not a distance 
between the other edge point P !b of this straight line 
and another straight line Ua is smaller than the refer- 
ence value. When the distance between the other 
edge point Pu> and the other straight line Us is smaller 

30 than the reference value, since both of the edge 
points Pb, and P )b are eventually positioned adjacent 
to the straight lines U? and Us. respectively, the proc- 
ess operation is advanced to a step S297. At this step 
S297, this line segment Li is registered into the sub- 

35 ject line segment list 

At a step S293, when it is so judged that the dis- 
tance between one edge point P b and the straight line 
L i7 is smaller than the reference value, otherwise 
when it is so judged at a step S294 that the distance 

40 between the other edge point P^ and the other 
straight line Ua is below than the reference value, the 
process operation is advanced to a step S295. At this 
step S295, a judgment is made as to whether or not 
a distance between the edge point P^and the straight 

4$ line U? is smaller than the reference value. When it is 
so judged that this distance is below than the refer- 
ence value, the process operation is further ad- 
vanced to a step S296. At this step S296, a check is 
made as to whether or not a distance between the 

so edge point P ta and the straight I ine Ue is smaller than 
the reference value. If this distance is smaller than the 
reference value, then the process operation is ad- 
vanced to a step S297 at which this straight line Li is 
registered into the subject line segment list 

55 When it is so judged at either the step S295, or 
the step S296 that the distance between the edge 
point P| b and the straight line U?. or the edge point P ta 
and the straight line Ue becomes largely different 
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from the reference value, and further when the 
straight line Li has been registered into the subject 
line segment list at the step S297, the operation proc- 
ess is advanced to a step S296 at which the variable 
V is incremented by 1. Then, the process operation 
is returned to a step S292, at which a similar process 
operation is repeated. 

In other words, when both one edge point and 
the other edge point P| b are positioned near the 
straight lines U 7 and this line segment is regis- 
tered as the processing subject into the subject line 
segment list 

In the embodiment of Fig. 47, one edge points 
P*i. P42. P43 of the straight lines L*. Uare arranged 
near the straight line U? respectively. Also, the other 
edge points P 51 , P52. P53 are arranged near the other 
straight line I,* respectively, so that the straight lines 
Li, L2. La are registered as the line segments to be 
processed. 

Next, a confirmation is made of a featured 
amount of a subject line segment as indicated in Fig. 
51 . That is, at a first step S311 , the head line segment 
Li in the subject line segment list is selected as a sub- 
ject to be processed. At a step S312, a check is done 
as to whether or not the data about this line segment 
Li is present. When it is judged that there is the data 
on the line segment U, a distance D| fl between the 
straight line L»7 and one edge point P ta of the line seg- 
ment Li is calculated at a step S313. At a step S314, 
a distance Dq> between the straight line L, 7 and the 
other edge point P| b is calculated. 

At a step S315, these two distances D b and Da 
calculated in the above-described manner are com- 
pared with each other. At a step S316 to a step S319, 
either the edge point within the edge points P| fl and 
Pib, which is located near the straight line U 7 , is stored 
as P| 7 , and the edge point located near the segment 
line Us is stored as Pib- 

Then, the process operation is advanced to a 
step S320 at which the variable T is incremented by 
1 . Thereafter, the process operation is returned to the 
step S312 at which a similar process operation is re- 
peated. 

As previously described, the coordinate of the 
both edges of the subject line segment are stored as 
the point corresponding to the straight line L, 7 , or the 
straight line Us located near these edge points. 

In the embodiment of Fig. 47, one edge points P 41 
to Ptt of the straight lines Li to U are stored as the 
edge point corresponding to the straight line U7, 
whereas the other edge points P51 to P53 are stored 
as the edge point corresponding to the other straight 
line Ua. 

A process operation shown in Fig. 52 is carried 
out as the calculation for the processed line segment 
data That is, at a first step S331, the head line seg- 
ment Li in the su bject line segment list is first selected. 
At a step S332, a confirmation is made that there is 


the data about the line segment Li. At the next step 
S333, the point P n which has been stored as the edge 
point corresponding to the straight line L»7 is moved 
on the straight line P. 7 . At a step S334, the point P® 

5 which has been stored as the edge point correspond- 
ing to the straight line L^a is moved on the straight line 
L^. Subsequently, the process operation is advanced 
to a step S335 at which the variable "i" is incremented 
by 1. Then, the process operation is returned to the 

10 step S332 at which a similar process operation is re- 
peated. 

Both of the edge points for the respective line 
segments Li are transported on the straight lines L.7 
and Ua in accordance with the above-described man- 
15 ner. 

I n the embodi ment of Fig. 48, the edge points P41 
and P 5 i, P42 and P52. P43 and Pss of both the edges of 
the respective line segments L 1t L3 are moved on 
either the straight line L,?. or the straight line Ls a , re- 
20 spectively. As a result, the edge points of the respec- 
tive straight lines are aligned. 

In order not to vary the directions of the respec- 
tive line segments Li to U, the respective edge points 
must be moved on the extended lines of these line 
25 segments. When the directions of the line segments 
are allowed to be changed, the respective edge 
points may be moved along an arbitrary direction 
such as the X-axis direction, the Y-axis direction, and 
the direction intersected with the reference lines U, 7 
30 and Us at a right angle. 

Fig. 53 to Fig. 56 represent such an embodiment 
that two straight lines intersect with each other at an 
arbitrary point. In this embodiment, as shown in Fig. 
53, an arc-shaped gesture Su is inputted so as to 
35 connect a line segment L, with a line segment L 11( 
which are to be processed. Furthermore, two ges- 
tures S12 and Si 3 are entered which are mutually 
crossed, and designate a cross point between these 
two line segments L, and Li 1, or extended lines there- 
to of. 

At a stepS341 of Fig. 57, a starting point Q 11b and 
an end point Q 11e of the arc-shaped gesture S 1n which 
has been entered in the above manner, are confirmed 
as illustrated in Fig. 54. At a step S342, a cross point 
4$ Qx between the gesture S 12 and the gesture S 13 is 
confirmed. 

Next, at a step S351 to a step S356 of Fig. 58, the 
retrieving process operation of the subject line seg- 
ment as defined atthe step S68 of Fig. 7 is performed. 

so In other words, at the first step S351, the head line 
segment Li of the established line segment list is to be 
processed. At a step S352, a check is done as t 
whether or not there is the data about the line seg- 
ment Li. When it is so judged that the data about the 

55 line segment Li is present the process operation is 
advanced to a step S353 at which another check is 
made as to whether or not a distance between the line 
segment Li and the starting point Qn. of the gesture 
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Sn is smaller than the reference value. 

When the distance is smaller than the reference 
value, the process operation is advanced to a step 
S355 at which this line segment Li is registered into 
the subject line segment list When the distance be- 
tween the line segment Li and the starting point Qn B 
is smaller than the reference value, the process op- 
eration is advanced to a step S354 at which a judg- 
ment is made as to whether or not a distance between 
this line segment Li and the end point Qn e is smaller 
than the reference value. When the distance from this 
end point Q 11o becomes smaller than the reference 
value, the process operation is advanced to a step 
S3S5 at which this line segment Li is registered into 
the subject line segment list 

After the line segment has been completely reg- 
istered into the su bject line segment list as described 
above, or when it is so judged at the step S353 and 
the step S364 that the respective distances between 
the starting point Qn, of the gesture Su and the line 
segment Li, and the end point Qn e thereof and this 
line segment Li are greater than the reference value, 
the process operation is advanced to a step S356. At 
this step, the variable T is incremented by 1. There- 
after, the process operation is returned to a step S352 
at which a similar process operation is repeated. 

As indicated in Fig. 54, the line segments Lu and 
Lt which are arranged near the starting point Q 11s of 
the gesture Su and the end point Q 11e are registered 
into the subject line segment list 

At a step S361 to a step S370 shown in Fig. 59, 
the process operation for confirming the featured 
amount of the subject line segment as defined at a 
step S81 of Fig. 8 is carried out At a first step S361 , 
the head line segment Li of the subject line segment 
list is to be processed. Then, at a step S362, a judg- 
ment is made as to whether or not the data on the I ine 
segment Li is present When it is so judged that there 
is the data on the line segment Li, the process oper- 
ation is advanced to a step S363 at which a direction 
(angle) "ai" of this line segment Li is obtained. 

Next, the process operation is advanced to a step 
S364 at which a point P h whose distance from the 
starting point Q 11v of the gesture S u becomes mini- 
mum and which is located on the line segment Li, is 
calculated. Furthermore, at a step S365, a distance 
between this point P tB and the starting point Q 11b is set 
as a distance D4 between this line segment Li and the 
starting point Qn. of the gesture Next the proc- 
ess operation is advanced to a step S366 at which a 
direction at. of the straight line Lis containing the point 
Pi. and a cross point Qx between the gestures S 12 and 
S 13 is obtained. 

Similarly, at a step S367, a point whose distance 
from the end point Q 11ft of the gesture becomes 
minimum, and which is located on the line segment U, 
is set as a "Pie u . Then, at a step S368, a distance be- 
tween this point Pie and the end point Qn 0 of the ges- 


ture Su is set as a distance Dei between the line seg- 
ment Li and the end point Qn *, Furthermore, at a step 

5369, a direction a lft of the straight line Lie containing 
the point Pie and the cross point Qx between the ges- 

5 ture Si 2 and S 13 is obtained. 

After the variable T is incremented by 1 at a step 

5370, the process operation is returned to a step 
S362 at which a similar process operation is repeat- 
ed. 

10 As described above, as shown in Fig. 55, both the 
direction an of the line segment Li 1, and the direction 
a1 of the line segment Li are obtained. It is assumed 
that such a point is Pes which is located on the line 
segment Ln positioned at the nearest position from 

15 the starting point Qn» of the gesture S 11p and a point 
is Pe2 which is present on the line Li and is located at 
the nearest position from the end point Qn 6 of the 
gesture S 11a Also, as shown in Fig. 56, a direction a» 
of the straight line Ls which passes through the point 

20 Pas on the straight line Li and the cross point Qx, is 
obtained. Further, a direction a» of the straight line Le 
passing through the point P e2 on the straight line Li 
and the cross point Qx is obtained. 

Moreover, the process operation for calculating 

25 the processed line segment data as defined at the 
step S82 of Fig. 8 is executed by performing process 
operations defined from a step S381 to a step S387 
shown in Fig. 60. That is, at a step S381, the head line 
segment Li in the subject line segment list is selected 

30 as a first processing subject At a step S362, a judg- 
ment is made as to whether or not there is the data 
about the line segment Li. 

When it is so judged that the data on the line seg- 
ment Li is present, the process operation is advanced 

35 to a step S363. At this step, a comparison is made for 
the distance D4 between the line segment Li and the 
starting point Q 1lB of the gesture S 11f with the distance 
D B) . 1 between this starting point Q 11ft and the line seg- 
ment U 1 positioned just before the line segment Li. 

40 Then, the straight line having the shorter distance is 
set as Ls, the direction ai therefore is set as As, and 
the point P to located at the nearest position from the 
starting point Q 11s of the gesture Si 1 is set as Ps. Fur- 
thermore, the direction ais of the straight line Lis con- 

45 taining this point Ps and the cross point Qx. 

Similarly, at a step S384, such a straight line 
whose distance Dei between the line segment Li and 
the end point Q 1l0 of the gesture Sn is minimum is set 
as Le. Also in this case, ai of this straight line is set 

50 as ae, Pie is set as Pe, and aie is set as a». There- 
after, the process operation is advanced to a step 
S385 at which the variable T is incremented by 1. 
Subsequently, the process operation is returned to a 
step S382 at which a similar process operation is re- 

55 peated. 

In other words, the process operation as defined 
at this step S382 to the step S385, the line segments 
located at the nearest positions from both of the start- 
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ing point Qn, and the ending point Qn B of the gesture 
Sit are selected as either the line segment Ls, or the 
line segment Le. 

Subsequently, the process operation is ad- 
vanced to a step S366 at which the straight line Ls lo- 
cated at the nearest position from the starting point 
Q«»of the gesture S u is rotated by an angle (a„ - aj 
around the point Ps as a centre which is located at the 
nearest position from the starting point Qn. and is lo- 
cated on this straight line Ls. That is, the straight line 
Ls is rotated in such a manner that either this line it- 
self, or the extended line thereof passes through the 
cross point Gx. Furthermore, at a step S387, the 
straight line Le located at the nearest position from 
the ending point Qn»of the gesture Sn is rotated by 
an angle (a„ - a e ) around the point Pe as a centre, 
which is located at the nearest position from the end 
point Q 11e . That is, the straight line Le is rotated in 
such a manner that either this straight line or the ex- 
tended line thereof passes through the cross point 
Qx. 

In Fig. 56, there is shown such that the straight 
line L ni is rotated around the point Pea as a centre, and 
is moved in such a manner that the extended line 
passes through the cross point Qx. Similarly, the 
straight line Li is rotated around the point Paa as a 
centre, and is moved and represented in such a way 
that this straight line Li passes through the cross 
point Qx 

Fig. 61 to Fig. 65 represent a further embodi- 
ment In this embodiment a gesture Si for arranging 
the abovedescribed line segments in an equiinterval; 
a gesture S 2 for vertically setting two straight lines; 
gestures S 7 and S a used to align both end points of a 
line segment; and gestures S e , S 5 , S fi used to arrange 
the line segments at a predetermined angle in a par- 
allel form are combined with each other and are en- 
tered. 

In this embodiment, the gesture S e is inputted to- 
gether with the gesture S 7 as a gesture for aligning 
both of the edge points, but also is employed in com- 
bination with the gestures S 5 and Se as a gesture for 
aligning the line segments at a preselected angle 
(namely, gesture S 4 shown in Fig. 37). As described 
above, a single gesture may be used as the construc- 
tive elements for the plural gestures. 

In this embodiment the gesture S 12 shown in Fig. 
62 corresponds to such a gesture that when the 
above-described respective gestures are inputted, 
this gesture is not executed but reserved, and when 
these entered gesture are finally and wholly execut- 
ed, the gesture S 12 is entered. 

When such a process operation is executed, the 
process operations as defined after the step S61 of 
Fig. 7 among the process operations indicated from 
Fig. 4 to Fig. 6, will be modified as illustrated in Fig. 
66 and Fig. 67. 

In other words, subsequent to the process oper- 


ation defined at either the step S41, or the step S48 
shown in Fig. 6, the process operation is advanced to 
a step S401 of Fig. 66, at which a check is made as 
to whether or not the gesture S 12 used to execute the 

5 reserve handwriting as shown in Fig. 62 is inputted. 
When it is so judged that the gesture S 12 for executing 
the reserve handwriting is not entered, the process 
operation is advanced to a step S402 at which a judg- 
ment is made as to whether or not there exists a plural 

10 i mage gesture su bject. Then, when it is so judged that 
there is such a plural image gesture subject, the proc- 
ess operation is further advanced to a step S403. At 
this step, the featured amount of the entered gesture 
is added and stored into the plural image featured 

15 handwriting amount memory unit 55, and then the 
process operation is returned to the step Si of Fig. 4. 
Then, a process operation is carried out for another 
image of the plural image gesture which will then be 
inputted. 

20 When it is so judged at the step S402 that there 
is no plural image gesture subject, the process oper- 
ation is accomplished. 

That is to say, the above-explained process op- 
eration is similar to the process operation defined in 
25 the case of Fig. 7. 

On the other hand, when it is so judged at the step 

5401 that the gesture S 12 to execute the reserve 
handwriting is inputted, the process operation is ad- 
vanced to a step S404. At this step, a judgment is 

so made as to whether or not the gesture subject is pres- 
ent in the list of the established gesture subject list 
memory unit 62. Then, when no gesture subject is 
present, the process operation is returned to the stop 

5402 at which the subsequent process operation is 
35 similarly performed. 

When it is so judged at the step S404 that there 
is the gesture subject in the established gesture sub- 
ject list the process operation is advanced to a step 
S405 at which a selection is made of such a gesture 
40 which owns the maximum point from these gesture 
subjects. Then, at a step S406, a featured amount of 
handwriting of the gesture is detected. As previously 
stated, this featured amount is required to change the 
display condition of the line segment under represen- 
ts tat ion. Then, the process operation is advanced to a 
step S404 at which the subject line segment list is ini- 
tialized. 

Next, the process operation is advanced tea step 
S408 at which a retrieve operations carried out for the 

so line segment to be processed by the gesture detected 
at the step S408 by using the featured amount of this 
detected gesture. At a step 409, a check is done 
whether or not there is a line segment to be process- 
ed. When it is so judged that no line segment to be 

55 processed is present the process operation is ad- 
vanced to a step S410 at which a check is done 
whether or not another gesture subject is present in 
the established gesture subject list When there is an- 
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other gesture subject, the process operation is ad- 
vanced to a step S411 at which a selection is made 
of another gesture subject having the second highest 
point The process operation is returned to the step 
$406 at which a similar process operation is executed 
by detecting a featured amount of this selected sub- 
ject 

On one hand, when it is so judged at the step 
S410 that there is no other gesture subject in the es- 
tablished gesture subject list the process operation 
is returned to the step S402 at which a judgment is 
done as to whether or not there is a plural image ges- 
ture subject In correspondence with this judgment re- 
sult a similar process operation to that of the above* 
explained case is executed. 

On the other hand, when it is so judged at the step 
S409 that there is such a line segment to be process- 
ed by the gesture command, the process operation is 
advanced to a step S421 at which a featured amount 
of this subject line segment is detected. Then, the 
process operation is advanced to a step S422 at 
which a calculation for the processed line segment 
data is performed. Subsequently, the process opera- 
tion is advanced to a step S423 at which a confirma- 
tion is made by a use r as to whether or not the gesture 
is executed. When the user does not allow the ges- 
ture execution, the process operation is returned to 
a step S410 at which such a process operation for 
checking whether or not another gesture subject is 
present in the established gesture subject list is per- 
formed. 

If the user allows to execute the gesture at the 
step S423, then the process operation is advanced to 
a step S424 at which the gesture handwriting is 
erased. At a step S425, the data about the line seg- 
ment is changed into the data about the processed 
line segment obtained at the step S422. As a conse- 
quence, the display conditions of the line segments 
are wholly changed. 

Next, the process operation is advanced to a step 
S426 to a step S429, at which the plural image ges- 
ture subject the established gesture subject the plu- 
ral image handwriting, and the plural image featured 
amount are detected. Then, the process operation is 
returned to the step S404 at which a similar process 
operation is repeated. 

When the above-explained process operations 
are earned out, as illustrated in Fig 63 to Fig. 65, the 
process operations corresponding to the inputted 
gestures S 1f S* S 5 , S&. $?, S 8 are wholly executed. 

That is, at the step 401 of Fig. 6, until the gesture 
Si 2 to execute the reserved handwriting is inputted, 
the process operations defined after the step S404 
are not carried out Namely, no process operation to 
carry out the previously entered gesture is carried 
out As a result the previously entered gesture is re- 
served as the reserved handwriting by performing 
the process operations defined before the step S401 . 


Then, after the gesture S 12 has been entered, these 
gestures of the reserved handwriting are wholly exe- 
cuted. 

Fig. 68 to Fig. 72 schematically represent a fur- 

5 ther embodiment In this embodiment a featured 
amount (parameter) may be more firmly designated 
by containing a character as a shape confirmation of 
inputted handwriting. To execute such a process op- 
eration, both of the process operation for detecting 

10 the featured amount of the gesture handwriting de- 
fined at the step S66, and the process operation for 
calculating the processed line segment data defined 
at the step S82 among the above-described process 
operations shown in Fig. 4 to Fig. 8 are performed as 

is represented in Fig. 73, or Fig. 74, respectively. 

It should be noted that the process operations de- 
fined at the step S68 and the step S81 are similarly 
carried out as shown in Fig. 24 or Fig. 25. 

In Fig. 73, at a first step S441, a confirmation is 

20 made of coordinate values for a starting point Pn. of 
a gesture Su and an end point Pn e thereof. At a step 
S442, both a gesture and a gesture S22. which are 
contained in a gesture S23 are confirmed, then, at a 
step S443 and a step 444, numeral values of the ges- 

25 ture S21 and the gesture S22 inside this gesture S23 
are interpreted. Then, at the final step S445, a numer- 
al value signified by combining the gesture S* with 
the gesture S22 is confirmed. 

In the embodiment shown in Fig. 68, a character 

30 of 60 is entered as the gestures S21 and S22 into the 
gesture having such a dosed shape that an inter- 
nal space is separated from an external space. Then, 
as shown in Fig. 69, after these gestures Su and S21 
to S23 have been inputted, when the reserved hand- 

35 writing executing gesture S 12 is entered, the charac- 
ter inside the gesture is confirmed, and this con- 
firmation result is displayed adjacent to this charac- 
ter, as represented in Fig. 70. 

On the other hand, at a step S451 of Fig. 74. the 

40 head line segment Li in the subject line segment list 
is set as a first processing subject. Then, at a step 
S452, a judgment is made as to whether or not the 
data about the line segment Li is present When the 
data on the line segment U is present the process op- 

45 eration is advanced to a step S453. At this step, a 
straight line is set as Ls, whose distance D,i from the 
starting point Qn B of the gesture Su s minimum, and 
a direction "ai" of this straight line Ls is set as "as". 
Further, a point P| B is set as Ps, which is located at the 

60 nearest position from the starting point Qn„ and is 
positioned on this straight line Ls. In other words, at 
the step S453, the distance obtained in the just 
before signal line U < l is compared with the distance 
D B jof the present line segment Li, and the straight line 

55 having the shorter distance is set as Ls. That is, under 
this condition, the straight line located at the nearest 
position from the starting point Qh. of the gesture Su 
is selected as the first line segment Ls to be process- 
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ed. 

Next, the process operation is advanced to a step 
S454 at which a straight line is set as Le whose dis- 
tance Dei from then end point Qn e of the gesture Su 
becomes minimum, and whose direction "ai a is set as 
"ae", and also a point Pie is set as Pe, which is located 
on this straight line and at the nearest position from 
the end point Qn e of the gesture Su. In other words, 
a comparison is performed between the present is 
performed between the present distance Dei and the 
distance D cM of the previously processed line seg- 
ment L M , and the straight line whose distance is 
small is set as Le. As a result, such a straight line lo- 
cated at the nearest position from the end point Q 1te 
of the gesture Sn is selected as the second subject 
line segment Le to be processed. 

Subsequently, the process operation is ad- 
vanced to a step S455 at which the variable "i" is in- 
cremented by 1 , and then the process operation is re- 
turned to the step S452 at which a similar process op- 
eration is repeated. 

At the step S452, when it is so judged that there 
is no data about the line segment Li (namely, retriev- 
ing operations for all data about line segment Li have 
been accomplished), the process operation is ad- 
vanced to a step S456. At this step, the direction "as" 
of the line segment Ls located at the nearest position 
from the starting point Qn. of the gesture Sn is com- 
pared with the direction "ae" of the line segment Le lo- 
cated at the nearest position from the end point Q 11o 
of this gesture Qn. When the direction "ae" is larger 
than the direction "as", the process operation is ad- 
vanced to a step S457 at which a judgment is made 
as to whether or not a value produced by subtracting 
"as" from "ae" is greater than 90°. 

When (ae-a,) is greater than 90°, the process op- 
eration is advanced to a step S458 at which a value 
of 160°-(ae-a.) is set to "angle". When (a e -aj is small- 
er than, or equal to 90°, the process operation is ad- 
vanced to a step S459 at which the value of (a e -a,) is 
directly set to "angle", that is, an acute angle (namely, 
smaller angle) among the angles defined between 
the line segment Ls and the line segment Le is set to 
"angle". 

Then, the process operation is advanced from 
either the step S458 or the step S459 to the step S460 
at which the line segment Ls located at the nearest 
position from the starting point Q 11b of the gesture Sn 
is rotated by an angle of ((a e -a,)-A 0 ) around the point 
Ps located at the nearest position from the starting 
point Q 11b of the gesture Sn as a centre. As a conse- 
quence, the line segment Ls located near the starting 
point Qn a of the gesture Sn is rotated by a preselect- 
ed angle with respect to the line segment Le located 
at the nearest position from the end point Q 11a , so that 
both of these line segments are adjusted to define an 
angle of A 0 . It should be understood that the value of 
"A" is such a value recognized at the step S445. 


When it is so judged at a step S456 that the angle 
"as" is greater than the angle "ae 0 , the process oper- 
ation is advanced to a step S461 at which a check is 
made as to whether or not the value of (a B -a a ) is larger 

5 than 90°. When (a.-a*) is larger than 90°, the process 
operation is advanced to a step S462 at which the val- 
ue of 1 80°-(a»-ae) is set to "angle". When the value of 
(a^-a 6 ) is smallerthan, or equal to 90°, the process op- 
eration is advanced to a step S463 at which the value 

10 of (a»-a,) is set to "angle". 

That is, similar to the above-described case, a 
smaller angle (acute angle) among the angles de- 
fined by the two straight lines Ls and Le is set to "an- 
gle". Then, at the step S464, the straight line Ls is ro- 
ts tated by A°-(a s -a^ around the point Ps as a centre, 
and the angle of this straight line Ls with respect to 
the straight line Le is adjusted to become A°. 

In the embodiment shown in Fig. 71 and Fig. 72. 
both of the direction as of the straight line La and the 

20 d irection a 12 of the straig ht line L 12 , which are located 
near the starting point and the end point of the ges- 
ture Sn, are obtained. Furthermore, as indicated in 
Fig. 72, the direction as of the straight line L 3 is 
changed, and the angle of this straight line L, with re- 

25 spect to the straight line L 12 is so corrected that it be- 
comes an angle A°(=60°) designated by the gesture 
S 2 i and the gesture S22. 

It should be noted that a plurality of windows may 
be utilized in order that the variation in the represen- 

30 tation made when the command corresponding to the 
gesture has been executed is represented to a user. 
When the execution of the input gesture is confirmed, 
not only such a YES/NO checking process, but also 
change/correct for the process subject/parameter 

35 may be employed. 

Not only the process operation for the presently 
available line segment, but also such a mode change 
as the gesture command recognizing/executing 
method (for example, interrupt of gesture entry, and 

40 allowable of using same handwriting by plural ges- 
tures) may be performed by the gesture commands. 

Although the display condition of the line seg- 
ment is corrected in the above-described embodi- 
ments, the present invention may be applied to such 

4$ cases that the display state of the character is 
changed and corrected. The latter-mentioned em- 
bodiment will now be described. 

Fig. 75 schematically represents a functional 
block diagram for a gesture execution process oper- 

so ation when a display condition of either a character, 
or a pattern (figure) is changed. The basic arrange- 
ment of this functional block diagram including a 
handwriting memory unit 171 through a processed 
pattern memory unit 182 is similar to the functional 

55 block diagram involving the handwriting memory unit 
71 to the processed line segment memory unit 82. In 
this embodiment, such a point that a pattern data 
base 170 is added is different from the functional 
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block diagram of Fig. 3. 

For instance, when the distance of the line seg- 
ment with respect to the above-described gesture 
handwriting is obtained, the distance between the 
gesture and the edge point of the line segment close 5 
to this gesture. When a distance between a pattern 
(character) and a gesture is obtained, a distance be- 
tween this gesture and a point located at the nearest 
position from an outer frame (circumscribed rectan- 
gle) of a character is obtained. 10 

Similarly, for instance, when a centre of the 
aboveexplained line segment is obtained, a centre 
between both edge points of the I ine segment. In case 
of a character, a central position thereof is not always 
clarified. Thus, a centre of an outer frame of a char- 15 
acter is recognized as a centre of this character. As 
the outer frame, a circle and a polygon other than a 
rectangle may be employed. When a normal pattern 
but not a character is processed, a gravity position 
thereof is recognized as a centre thereof. As descri- 20 
bed above, since the different process operations are 
required for each of objects such as characters and 
patterns, these different process operations are per- 
formed based on a pattern data base 170, and the 
characters and patterns are supplied to a process de- 25 
termined. 

The process determining unit 175 reads out such 
data as the subject pattern retrieving method, the 
subject pattern featured amount extracting method, 
and the subject pattern processing method cone- 30 
sponding to the gestures stored in the complete ges- 
ture subject 1st memory unit 173 from a gesture data 
base 1 72, and also reads out a featured amount of a 
gesture enter from the handwriting featured amount 
memory unit 1 74. Then, the process operation is de- 35 
termined in correspondence with these datatooontrol 
a subject pattern retrieving unit 177, a featured 
amount extracting unit 179, and a pattern processing 
unit 181. 

In an established pattern list memory unit 176, a 40 
pattern sort number is stored in addition to the previ- 
ously inputted data about the patterns (patterns to be 
processed) other than the gesture. This data is proc- 
essed in accordance with the above-explained nec- 
essary modification process operation by the pattern 45 
data base 1 70, and thereafter is supplied to the proc- 
ess determining unit 175. The process determining 
unit 1 75 determines the process operation suitable for 
the sort of the gesture by utilizing this pattern sort 
number with respect to the element of the list so 

Under control of the process determining unit 
175, the subject pattern retrieving unit 177 retrieves 
a pattern to be processed from the line segments 
stored in the established pattern list memory unit 1 76, 
and supplies the retrieved result to the subject pat- 55 
tern list memory unit 178 so as to be stored therein. 

Under control of the process determining unit 
1 75, the featured amou nt extracting unit 1 79 extracts 


the featured amount of the subject pattern stored in 
the subject pattern list memory unit 1 78, and supplies 
the extracted result to the pattern featured amount 
memory unit 180 to be stored therein. The pattern 
processing unit 181 processes the featured amount of 
the pattern to be processed, which has been stored 
in the pattern featured amount memory unit 180, and 
produces data used to change the display condition 
of this pattern under control of the process determin- 
ing unit 175. Then, this data is furnished to the proc- 
essed pattern memory unit 182 so as to be stored 
therein. 

The data stored in this processed pattern mem- 
ory unit 182 is read and then supplied via the display 
controller 11 to the V-RAM 12 to be expanded as bit 
map data. Then, this expanded data is read by the dis- 
play controller 11 to be outputted on the display unit 
13 for a representation purpose. 

Fig. 76 to Fig. 80 schematically indicate an em- 
bodiment such that a character is aligned on a prese- 
lected straight line, to realize such an embodiment 
the process operation defined at the step S66 in Fig. 
7 is earned outas shown in a step S501 of Fig. 81 . that 
is, as shown in Fig. 76, when such a gesture S*i is in- 
putted which is drawn by bending at a substantially 
right angle one edge portion of a sufficiently long 
straight line at a short portion thereof, as illustrated 
in Fig. 77, a straight line Usi containing a longer line 
segment among this gesture S31 is detected. 

It should be noted that when this gesture S31 has 
such a shape with a bending angle at a substantially 
right angle, and one length of which is longer than the 
other length multiplied by a reference value, this ges- 
ture is recognized as a gesture for a character align- 
ment 

The process operation for retrieving the subject 
line segment as defined at the step S68 of Fig. 7 is 
carried out as illustrated in Fig. 82. That is, the head 
pattern Li in the established pattern list is selected as 
a first processing subject Since a line segment Li, a 
character Lei, or a pattern LFi has been stored as the 
pattern Li in the established pattern list any one of 
them is selected. At the next step S512, a judgment 
is made as to whether or not there is data about the 
pattern Li. When the data about the pattern Li is pres- 
ent, the process operation is advanced to a step S51 3 
at which it is so judged whether or not the distance be- 
tween the straight Us, and the pattern Li is smaller 
than a preset reference value. When the pattern Li is 
positioned close to the straight line U31, the process 
operation is advanced to a step S514 at which this 
pattern Li is registered in the subject pattern list 

Either when it is so judged at the stop S51 3 that 
the pattern Li is relatively largely separated from the 
straight line Uai, or after the pattern Li has been reg- 
istered into the subject pattern list at the step S514, 
the process operation is advanced to a step S515. At 
this step S515, the variable Y Is incremented by 1, 
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and the process operation is returned to the step 
S512 at which a similar process operation is repeat- 
ed. The pattern (character) of the gesture S31 to be 
processed is retrieved by repeating the above- 
described process operation. 5 

In case of the embodiment shown in Fig. 76 to 
Fig. 60, there are characters G| to C 8 as the charac- 
ters arranged near the straight line L^. As a result, 
in this case, these 8 characters are selected as the 
subject processed by the gesture S31. 

Subsequently, the process operation for confirm- 
ing the featured amount of the line segment to be 
processed at the step S61 shown in Fig. 8 is carried 
out as shown at step SS21 to S524 of Fig. 83. that is, 
at a first step S521 , the head pattern Li in the subject 
pattern list is first set as a subject to be processed. 
At a step S522, a confirmation is made as to whether 
or not there is data about this pattern Li. When there 
is the data about the pattern Li, the process operation 
is advanced to a step S523 at which a reference point 
Pi of this pattern Li is obtained. This reference point 
Pi is defined as, for instance, a lower right point of an 
outer frame of the pattern. 

Thereafter, the variable T is incremented by 1 at 
the step S524, and then the process operation is re- 
turned to the step S522 at which a similar process op- 
eration is repeated. Thus, the point located at the 
nearest position from the straight line Lrti corre- 
sponding to the gesture S 31 corresponding to the ges- 
ture within the outer frames of the respective pat- 
terns Li is stored as "Pi". 

In the embodimentshown in Fig. 79, the points Pi 
to P a located at the right lower positions of the char- 
acters C| and C e are stored as the points correspond- 
ing to the straight line for the gesture S31. 

The process operation for calculating the proc- 
essed line segment data at the step S82 of Fig. 8 is 
executed as illustrated at steps S531 to S534 of Fig. 
84. Namely, at a first step SS31, the head pattern Li 
of the subject pattern list is selected, and at a step 
S532, a confirmation is made as to whether or not 
there is data about this pattern Li. When the data 
about the pattern Li is present, this point Pi is moved 
on the straight line U1 corresponding to the gesture 
S31 at a step S533. Then, the process operation is ad- 
vanced to a step S534 at which the variable V is in- 
cremented by 1. Then, the process operation is re- 
turned to the step S532 at wh ich a si mi lar process op- 
eration is repeated. Thus, the point of the character to 
be processed is moved on the straight line U31. 

In the embodimentshown in Fig. 80, the points Pi 
to P e of the outer frame of the characters to C e are 
transported on the straight line L^. 

It should be understood that althoug h the point Pi 
is transported on the straight line U31 in this embodi- 
ment, this point Pi may be alternatively transported as 
follows: The point Pi may be moved on the straight 
line in parallel thereto in such a manner that a sum- 


mation of the distances between the straight line L^ 
and this point Pi becomes minimum. The point Pi may 
be moved on the straight line in parallel thereto 
in such a manner that a powered summation of the 
distances between this point Pi and the straight line 
1^ becomes minimum. 

As previously explained, in accordance with the 
handwriting input display method as recited in claim 
1 , since the display condition of the line segment is 
changed in correspondence with the inputted ges- 
ture, it is possible to quickly and surely change the 
display condition of the line segment 

Also, in accordance with the handwriting input 
display method as recited in claim 2, since the display 
condition of the shape is changed in correspondence 
with the featured amount of the gesture, the display 
condition can be more precisely changed by investi- 
gating the input condition of the gesture. 


Claims 

1. A handwriting input display apparatus compris- 
ing: 

display means for displaying either a line 
segment or a character, corresponding to a calcu- 
lation result of a computer 

tablet means, at least a portion of which is 
transparent, and formed on said display means, 
for outputting coordinate data in response to an 
operation; 

operation means for operating said tablet 
means; 

at least one input handwriting storing 
means for storing input handwriting contracted of 
said coordinate data output from said table 
means in response to the operation of said oper- 
ation means; 

means for detecting a featured amount 
from the handwriting stored in said input hand- 
writing storing means; 

means for comparing said featured 
amount with a featured amount of a gesture stor- 
ed in a previously employed gesture data base; 
and 

means for specifying the operation of said 
operating means as a gesture when said featured 
amount is similar to the feature amount of said 
gesture at a level more than a predetermined lev- 
el, thereby executing the corresponding gesture; 
wherein: 

under such a condition that either the line 
segment, or the character is being displayed on 
said display, said featured amount is entered by 
performing a process operation for said line seg- 
ment, or said character by way of said operating 
means and in response to the entry of said fea- 
tured amount a preselected command is per- 
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formed to change the display condition of said 
line segment, or said character. 

2. A handwriting input display apparatus as claimed 

in claim 1 , wherein under such a condition that a s 
plurality of line segments are displayed on said 
display, when said operation corresponds to such 
an operation for bridging at least three line seg- 
ments, said command is such a command for re- 
arranging at least said three line segments at an 10 
equidistant interval to be displayed. 

3. A handwriting input display apparatus as claimed 
in claim 1 or 2, wherein under such a display con- 
dition that a plurality of line segments are dis- 
played, when said operation corresponds to an 
operation to designate an intersecting angle be- 
tween at least two line segments, said command 
corresponds to such a command to rearrange at 
least said two line segments at a predetermined 
angle for representation. 

4. A handwriting input display apparatus as claimed 
in claim 3 wherein when said operation is such an 
operation to designate intersection on a prese- 
lected point on each of said at least two line seg- 
ments, said command corresponds to such a 
command to rearrange said at least two line seg- 
ments at a predetermined point on each of said 
line segments at a preselected angle for repre- 
sentation. 

5. A handwriting input display apparatus as claimed 
in claim 3 or claim 4 wherein when said operation 
is such an operation to enter a numeral number 
adjacent to said at least two line segment, said 
command corresponds to a command to rear- 
range said at least two line segments at an angle 
designated by said numeral number. 

6. A handwriting input display apparatus as claimed 
in any one of claims 1 to 5, wherein under such a 
condition that a plurality of line segments are dis- 
played on said display, when said operation cor- 
responds to such an operation to cause said at 
least two line segments to intersect with each 
other adjacentto edge points of said two line seg- 
ments, said command corresponds to such a 
command to rearrange that edge points of at 
least two line segments are aligned on the same 
straight line. 

7. A handwriting input display apparatus as claimed 
in claim 6, wherein when said operation corre- 
sponds to such an operation to cause said atleast 
two line segments to intersect with each other at 
a point near the other edge points of said line seg- 
ments, said command is such a command to re- 


arrange that both edge points of said at least two 
line segments are arranged on the same two 
straight lines for representation. 

8. A handwriting input display apparatus as claimed 
in any one of claims 1 to 7, wherein under such a 
condition that a plurality of line segments are dis- 
played on said display, when said operation is 
such an operation to enter parallel lines at a pre- 
determined angle with respect to said at least two 
line segments, said command corresponds to 
such a command to rearrange that said at least 
two line segments are arranged in parallel to 
each other. 

9. A handwriting input display apparatus as claimed 
in claim 8 wherein when said operation further 
contains an operation for causing said at least 
two line segments to intersect with each other, 
said command corresponds to such a command 
to rearrange that cross points on said at least two 
line segments are arranged on the same straight 
line. 

10. A handwriting input display apparatus as claimed 
in anyone of claims 1 to 9, wherein when said op- 
eration further contains such an operation to tem- 
porarily stop the execution of said command, the 
execution of said command is temporarily stop- 
ped until release operation is performed. 

11. A handwriting in put display apparatus as claimed 
in any one of claims 1 to 1 0, wherein under such 
a condition that a plurality of characters are dis- 
played on said display, when said operation cor- 
responds to such an operation for entering a 
straight line with a predetermined relationship 
with respect to said at least two characters, said 
command corresponds to such a command that 
said at least two characters are aligned to be re- 
arranged for representation. 
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step S82 is started 
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step S82 is ended 


EP0644476A2 


FIG. 53 



A4 


EPO 644476 A2 


FIG. 55 



CO 


EP 0644 476 A2 


FIG. 57 


step S66 is started 


confirm storting point (Qm) and end point (Qm) 
of input handwriting Sb 
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confirm coordinate Ox of cross points of input 
handwriting Se, So 
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step 566 is ended 
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step 581 is started 
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* 
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step S81 is ended 
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step S82 is started 
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NO 


{Is there data on line segment (Li) 7 


S382 


YES 


S383 


set straight line Li of small Dn as U its a as a*, 
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add input featured amount plural 
image featured amount 
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step S66 is started 


confirm coordinates of storting point (Qm) and 
end point (Q*) of input handwriting Sn 
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confirm handwriting (821,522) contained in input handwriting Sn 







interpret input handwriting 5a as numeral (n1=6) 
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interpret input handwriting S22 as numeral (n2=0) 
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step S66 is ended 
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step 582 is started 
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\of subject line segment list A 
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step S66 is started 
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confirm straight line Usi containing long line 
segment portion of input handwriting Sn 


step S66 is ended 


EP 0644 476 A2 


FIG. 82 
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step S81 is ended 
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step S82 is started 
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